S

S3 dataset

Data sources & feeders available

Stream a dataset object from S3 with a pure-Rust SigV4 signer.

Install
$ loadr plugin install s3-dataset
examples/plugins/s3-dataset.yaml
# Drive a run from a CSV/JSONL dataset stored in S3, via the
# `loadr-plugin-s3-dataset` native SERVICE plugin (data-source / feeder role).
#
# The plugin is NOT built into loadr core. At the start of the run it fetches a
# single object from S3 over HTTPS, signs the GET with AWS Signature V4 (a
# pure-Rust signer — no AWS SDK, no OpenSSL) and parses the body into rows. The
# rows become an ordinary `data:` feeder: each column is referenced with the
# usual `${data.<name>.<column>}` interpolation, exactly like a local
# `type: csv` file, but the fixture lives in a bucket next to the system under
# test. S3 is touched ONCE, at startup — never on the request hot path.
#
# Build + install the plugin, then run:
#   cargo build -p loadr-plugin-s3-dataset --release
#   mkdir -p dist && cp plugins/loadr-plugin-s3-dataset/plugin.toml dist/ \
#     && cp target/release/libloadr_plugin_s3_dataset.so dist/
#   loadr plugin install dist
#   AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... \
#     loadr run examples/plugins/s3-dataset.yaml
#
# Or point the plan's `plugins:` entry at the built artifact directly (below).
name: s3-dataset
description: replay a signup flow from a users dataset held in S3

plugins:
  # Resolve `s3-dataset` by name from the plugins dir (after `loadr plugin install`).
  # To run straight from a build tree instead, set:
  #   path: target/release/libloadr_plugin_s3_dataset.so
  - name: s3-dataset

data:
  users:
    service: s3-dataset          # this feeder is backed by the service plugin
    config:
      # Either a full `url:`, or `bucket` + `key` + `region` (virtual-hosted),
      # or add `endpoint:` for an S3-compatible store (MinIO, path-style).
      bucket: my-loadtest-fixtures
      key: datasets/users.csv
      region: eu-west-2
      format: auto               # csv | jsonl | auto (by key extension)
      has_header: true           # csv only
      # Credentials come from config or the standard AWS_* env vars. Empty
      # interpolations are ignored, so an unset session token is fine.
      access_key_id: "${env.AWS_ACCESS_KEY_ID}"
      secret_access_key: "${env.AWS_SECRET_ACCESS_KEY}"
      session_token: "${env.AWS_SESSION_TOKEN}"
    mode: shared                 # all VUs share one cursor (per_vu: each VU its own)
    pick: sequential             # sequential | random | shuffle
    on_eof: recycle              # wrap around (stop: retire the VU at end of set)

scenarios:
  replay_signups:
    executor: constant-vus
    vus: 25
    duration: 2m
    flow:
      - request:
          name: fetch profile
          method: GET
          url: "https://api.example.com/users/${data.users.id}"
          headers: { X-User-Email: "${data.users.email}" }
          checks:
            - { type: status, equals: 200 }
            - { type: duration, name: profile is fast, max: 400ms }

thresholds:
  checks: [ "rate>0.99" ]
  http_req_duration: [ "p(95)<400ms" ]

A real run: install from the signed index, then watch the plugin work.

A runtime plugin, never in the binary

Installing pulls a per-platform driver from the signed index, verifies its SHA-256 and checks its ABI before it ever loads. Remove it any time with loadr plugin remove s3-dataset.