Information Crawler
v1.12.12
Sign InAppBlogDocs
Getting Started
  • Getting Started
Projects
  • Create a Project
Sources
  • Add an RSS Source
  • Crawl Content
Pipelines
  • Create a Pipeline
Publication
  • Material Package to Mailoo
Home/User Guide/Pipelines

Create a Pipeline

On this page

  • Create the pipeline
  • Add a processing rule
  • Run

A pipeline runs processing rules in order: crawl sources, then transform, generate or publish content.

Create the pipeline

  1. Open Pipelines and click Add Pipeline (or Add Pipeline on the project page).
  2. Enter Name, choose Project and Schedule Type: Manual, Interval or Cron.
  3. Click Create Pipeline.

Create pipeline form

Add a processing rule

  1. On the pipeline page, click Configure Processing, then add a rule.
  2. Step 1: enter Rule Name, choose Processing Type (for collecting content --- Crawling) and keep the suggested Execution Order. Click Next: Configure Rule.
  3. Step 2: on the Processing tab choose the source to crawl, then click Create Processing Rule.

Create processing rule

Run

Click Execute Pipeline. Each run appears in Tasks with its status. A failed task shows the reason --- for example, a crawling rule without a source.

Pipeline page

← Crawl ContentMaterial Package to Mailoo →