logoRocketFlow

Auto-Retrain

Automatically re-crawl URL-based knowledge sources to keep your chatbot up to date

Auto-Retrain

Keep your chatbot's knowledge current by automatically re-crawling URL sources on a schedule.

How It Works

  1. A scheduled background task checks for URL-type sources with auto-retrain enabled (the schedule is attached in the background-job dashboard, typically hourly)
  2. Checks if enough time has passed since the last retrain (default: 24 hours; a source that has never been retrained runs immediately)
  3. Re-crawls the URL
  4. Compares word count — if content changed by more than 10 words, re-indexes the source
  5. If unchanged, skips re-indexing (saves processing)

Sources with auto-retrain enabled show a refresh icon in the Knowledge sources list.

Configuration

  • Interval: 24 hours (configurable per source via retrainInterval, in hours)
  • Content change detection: Compares word counts before and after crawl
  • Skips unchanged content: Only re-indexes when content actually changed
  • Applies to url, website, and webpage source types

Last updated on