S3

​

What It Does

• reads files from one or more buckets
• converts content to Markdown for indexing
• runs ingestion on configurable schedules

​

Required Environment Variables

• S3_ACCOUNT1_ENDPOINT: S3 API endpoint URL
• S3_ACCOUNT1_ACCESS_KEY: access key ID
• S3_ACCOUNT1_SECRET_KEY: secret key
• S3_ACCOUNT1_REGION: bucket region
• S3_ACCOUNT1_USE_SSL: True or False
• S3_ACCOUNT1_BUCKETS: bucket name(s), comma-separated for multiple
• S3_ACCOUNT1_SCHEDULES: ingestion interval(s) in seconds (default is 3600)

​

config.yaml Example

sources:
  - type: "s3"
    name: "account1"
    enabled: true  # optional, default: true
    config:
      endpoint: "${S3_ACCOUNT1_ENDPOINT}"
      access_key: "${S3_ACCOUNT1_ACCESS_KEY}"
      secret_key: "${S3_ACCOUNT1_SECRET_KEY}"
      region: "${S3_ACCOUNT1_REGION}"
      use_ssl: "${S3_ACCOUNT1_USE_SSL}"
      buckets: "${S3_ACCOUNT1_BUCKETS}"
      schedules: "${S3_ACCOUNT1_SCHEDULES}"

​

Configuration Reference

​

Multiple S3 Accounts