144 lines
4.9 KiB
Markdown
144 lines
4.9 KiB
Markdown
# beam
|
|
|
|
File transfer service via http. Upload with `curl` or a browser, share the
|
|
returned URL, and the file is deleted automatically once it expires.
|
|
|
|
## Usage
|
|
|
|
Upload a single file with `PUT`; the response body is the download URL:
|
|
|
|
```sh
|
|
curl --upload-file myfile.txt https://beam.example.com/myfile.txt
|
|
# https://beam.example.com/x123abcx/myfile.txt
|
|
```
|
|
|
|
Upload one or more files as a multipart form. This is the endpoint the web UI
|
|
uses, and it responds with JSON:
|
|
|
|
```sh
|
|
curl -F 'files=@a.txt' -F 'files=@b.txt' https://beam.example.com/upload
|
|
# {"urls":["https://beam.example.com/x123abcx/a.txt", "..."]}
|
|
```
|
|
|
|
If some parts could not be stored, their names are returned alongside the
|
|
successful URLs:
|
|
|
|
```json
|
|
{"urls": ["https://beam.example.com/x123abcx/a.txt"], "failed": ["b.txt"]}
|
|
```
|
|
|
|
Errors use a consistent shape:
|
|
|
|
```json
|
|
{"error": "Upload exceeds maximum size of 5368709120 bytes"}
|
|
```
|
|
|
|
Download a file, or just open the URL in a browser:
|
|
|
|
```sh
|
|
curl https://beam.example.com/x123abcx/myfile.txt -o myfile.txt
|
|
```
|
|
|
|
Downloads support range requests, so an interrupted transfer can be resumed:
|
|
|
|
```sh
|
|
curl -C - -O https://beam.example.com/x123abcx/myfile.txt
|
|
```
|
|
|
|
### Endpoints
|
|
|
|
| Method | Path | Purpose |
|
|
| ------------ | -------------------- | ---------------------------------------- |
|
|
| `GET` | `/` | Web UI with usage instructions |
|
|
| `PUT` | `/{filename}` | Upload one file, returns the URL as text |
|
|
| `POST` | `/upload` | Multipart upload, returns JSON |
|
|
| `GET`/`HEAD` | `/{id}/{filename}` | Download, supports `Range` |
|
|
| `GET` | `/healthz` | Liveness/readiness probe |
|
|
|
|
## Configuration
|
|
|
|
Configuration is via environment variables only. Invalid values are rejected at
|
|
startup.
|
|
|
|
| Variable | Default | Description |
|
|
| ------------------ | ---------- | ----------------------------------------------------------------------- |
|
|
| `UPLOAD_DIR` | `/uploads` | Where uploaded files are stored. |
|
|
| `PORT` | `8080` | Port to listen on. |
|
|
| `FILE_DURATION` | `168h` | How long a file is kept. Set to `0` to disable cleanup entirely. |
|
|
| `DRYRUN_CLEAN` | `true` | When true, cleanup only logs what it *would* delete. See the note below. |
|
|
| `CLEANUP_INTERVAL` | `6h` | How often the cleanup sweep runs. |
|
|
| `MAX_UPLOAD_SIZE` | `5368709120` | Maximum bytes per request (5 GiB), covering all files in one upload. |
|
|
| `SHUTDOWN_GRACE` | `15s` | How long to wait for in-flight requests on SIGTERM. |
|
|
|
|
> **`DRYRUN_CLEAN` defaults to `true`.** Until you set `DRYRUN_CLEAN=false`,
|
|
> expired files are never actually deleted — the service only logs
|
|
> `Dry run: Would remove expired file ...` while the disk keeps filling up.
|
|
|
|
Durations use Go's `time.ParseDuration` syntax (`30m`, `12h`, `168h`).
|
|
`MAX_UPLOAD_SIZE` is a plain byte count.
|
|
|
|
Notes on behaviour worth knowing:
|
|
|
|
- `MAX_UPLOAD_SIZE` limits the **whole request**, so a multipart upload of
|
|
several files is measured by their combined size, not per file.
|
|
- A single multipart request may contain at most 100 files.
|
|
- Expiry is based on the file's modification time; downloading does not extend
|
|
a file's life.
|
|
- `SHUTDOWN_GRACE` should be aligned with the pod's
|
|
`terminationGracePeriodSeconds`. If an upload is still running when the grace
|
|
period elapses, remaining connections are closed and the process still exits
|
|
cleanly.
|
|
|
|
## Running
|
|
|
|
```sh
|
|
docker run -p 8080:8080 \
|
|
-e DRYRUN_CLEAN=false \
|
|
-e FILE_DURATION=168h \
|
|
-v /srv/beam-uploads:/uploads \
|
|
docker.sdal.tech/beam:latest
|
|
```
|
|
|
|
The container runs as the non-root user `beam` (uid 1000), which must be able
|
|
to write to whatever is mounted at `/uploads`.
|
|
|
|
## Development
|
|
|
|
```sh
|
|
go build ./... && go vet ./... && gofmt -l . && go test ./...
|
|
```
|
|
|
|
Run locally:
|
|
|
|
```sh
|
|
UPLOAD_DIR=/tmp/beam-uploads PORT=8080 DRYRUN_CLEAN=false go run .
|
|
```
|
|
|
|
Note that without a reverse proxy setting `X-Forwarded-Host`, generated
|
|
download URLs fall back to `localhost:8080`. In production beam runs behind a
|
|
TLS-terminating ingress that sets `X-Forwarded-Proto` and `X-Forwarded-Host`,
|
|
which is where the URLs come from.
|
|
|
|
## Versioning
|
|
|
|
`build.sh` owns the application version via the `VERSION` variable and passes
|
|
it to the Docker build as a build argument. Builds append the current git short
|
|
hash with `GIT_SHA` so the homepage can display the exact image version near the
|
|
bottom-left edge.
|
|
|
|
```sh
|
|
./build.sh
|
|
```
|
|
|
|
To push `docker.sdal.tech/beam:latest` after building:
|
|
|
|
```sh
|
|
./build.sh push
|
|
```
|
|
|
|
Local non-Docker builds display `vdev` unless a version is injected manually:
|
|
|
|
```sh
|
|
go build -ldflags "-X main.version=0.1.0-dev" -o beam .
|
|
```
|