noselasd/beam
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add Improve UI error handling, add download resuming, improve cleanup

Browse Source 
code
This commit is contained in:
noselasd
2026-08-30 14:30:05 +02:00
parent 87dc427db1
commit 28537b2591
6 changed files with 825 additions and 207 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.dockerignore
+7
View File
@@ -4,3 +4,10 @@ uploads/
.gitignore
Dockerfile
.dockerignore
beam
*_test.go
.vscode/
.github/
.clinerules
CLAUDE.md
.claude/
.gitignore
+1
View File
@@ -4,3 +4,4 @@ beam
*.dll
*.so
*.dylib
.vscode/
Dockerfile
+1 -1
View File
@@ -27,7 +27,7 @@ RUN apk add --no-cache mailcap \
USER beam
WORKDIR /app
COPY --from=builder /build/index.html.tmpl /build/beam .
COPY --from=builder /build/beam .
EXPOSE 8080
README.md
+122 -2
View File
@@ -1,10 +1,130 @@
# beam
File transfer service via http.
File transfer service via http. Upload with `curl` or a browser, share the
returned URL, and the file is deleted automatically once it expires.
## Usage
Upload a single file with `PUT`; the response body is the download URL:
```sh
curl --upload-file myfile.txt https://beam.example.com/myfile.txt
# https://beam.example.com/x123abcx/myfile.txt
```
Upload one or more files as a multipart form. This is the endpoint the web UI
uses, and it responds with JSON:
```sh
curl -F 'files=@a.txt' -F 'files=@b.txt' https://beam.example.com/upload
# {"urls":["https://beam.example.com/x123abcx/a.txt", "..."]}
```
If some parts could not be stored, their names are returned alongside the
successful URLs:
```json
{"urls": ["https://beam.example.com/x123abcx/a.txt"], "failed": ["b.txt"]}
```
Errors use a consistent shape:
```json
{"error": "Upload exceeds maximum size of 5368709120 bytes"}
```
Download a file, or just open the URL in a browser:
```sh
curl https://beam.example.com/x123abcx/myfile.txt -o myfile.txt
```
Downloads support range requests, so an interrupted transfer can be resumed:
```sh
curl -C - -O https://beam.example.com/x123abcx/myfile.txt
```
### Endpoints
| Method | Path | Purpose |
| ------------ | -------------------- | ---------------------------------------- |
| `GET` | `/` | Web UI with usage instructions |
| `PUT` | `/{filename}` | Upload one file, returns the URL as text |
| `POST` | `/upload` | Multipart upload, returns JSON |
| `GET`/`HEAD` | `/{id}/{filename}` | Download, supports `Range` |
| `GET` | `/healthz` | Liveness/readiness probe |
## Configuration
Configuration is via environment variables only. Invalid values are rejected at
startup.
| Variable | Default | Description |
| ------------------ | ---------- | ----------------------------------------------------------------------- |
| `UPLOAD_DIR` | `/uploads` | Where uploaded files are stored. |
| `PORT` | `8080` | Port to listen on. |
| `FILE_DURATION` | `168h` | How long a file is kept. Set to `0` to disable cleanup entirely. |
| `DRYRUN_CLEAN` | `true` | When true, cleanup only logs what it *would* delete. See the note below. |
| `CLEANUP_INTERVAL` | `6h` | How often the cleanup sweep runs. |
| `MAX_UPLOAD_SIZE` | `5368709120` | Maximum bytes per request (5 GiB), covering all files in one upload. |
| `SHUTDOWN_GRACE` | `15s` | How long to wait for in-flight requests on SIGTERM. |
> **`DRYRUN_CLEAN` defaults to `true`.** Until you set `DRYRUN_CLEAN=false`,
> expired files are never actually deleted — the service only logs
> `Dry run: Would remove expired file ...` while the disk keeps filling up.
Durations use Go's `time.ParseDuration` syntax (`30m`, `12h`, `168h`).
`MAX_UPLOAD_SIZE` is a plain byte count.
Notes on behaviour worth knowing:
- `MAX_UPLOAD_SIZE` limits the **whole request**, so a multipart upload of
several files is measured by their combined size, not per file.
- A single multipart request may contain at most 100 files.
- Expiry is based on the file's modification time; downloading does not extend
a file's life.
- `SHUTDOWN_GRACE` should be aligned with the pod's
`terminationGracePeriodSeconds`. If an upload is still running when the grace
period elapses, remaining connections are closed and the process still exits
cleanly.
## Running
```sh
docker run -p 8080:8080 \
-e DRYRUN_CLEAN=false \
-e FILE_DURATION=168h \
-v /srv/beam-uploads:/uploads \
docker.sdal.tech/beam:latest
```
The container runs as the non-root user `beam` (uid 1000), which must be able
to write to whatever is mounted at `/uploads`.
## Development
```sh
go build ./... && go vet ./... && gofmt -l . && go test ./...
```
Run locally:
```sh
UPLOAD_DIR=/tmp/beam-uploads PORT=8080 DRYRUN_CLEAN=false go run .
```
Note that without a reverse proxy setting `X-Forwarded-Host`, generated
download URLs fall back to `localhost:8080`. In production beam runs behind a
TLS-terminating ingress that sets `X-Forwarded-Proto` and `X-Forwarded-Host`,
which is where the URLs come from.
## Versioning
The Dockerfile owns the base application version with the `VERSION` build argument. Docker builds append the current git short hash with `GIT_SHA` so the homepage can display the exact image version near the bottom-left edge.
`build.sh` owns the application version via the `VERSION` variable and passes
it to the Docker build as a build argument. Builds append the current git short
hash with `GIT_SHA` so the homepage can display the exact image version near the
bottom-left edge.
```sh
./build.sh
index.html.tmpl
+273 -56
View File
@@ -6,15 +6,19 @@
<title>Beam - File Upload Service</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
html { color-scheme: light; }
body {
font-family: monospace;
max-width: 800px;
margin: 40px auto;
padding: 20px;
line-height: 1.6;
background: #fff;
color: #222;
}
h1 { margin-bottom: 30px; font-size: 2em; }
h2 { margin: 30px 0 15px 0; font-size: 1.3em; }
h3 { margin: 0 0 10px 0; font-size: 1.05em; }
.section {
background: #f5f5f5;
padding: 20px;
@@ -30,12 +34,13 @@
border-radius: 4px;
overflow-x: auto;
}
.note { margin-top: 10px; }
.upload-area {
border: 3px dashed #ccc;
padding: 40px;
text-align: center;
cursor: pointer;
transition: all 0.3s;
transition: border-color 0.3s, background 0.3s;
}
.upload-area.dragover {
border-color: #333;
@@ -44,6 +49,12 @@
.upload-area:hover {
border-color: #666;
}
.upload-area.disabled {
cursor: not-allowed;
opacity: 0.6;
}
.upload-area.disabled:hover { border-color: #ccc; }
.hint { margin-top: 10px; font-size: 0.85em; color: #666; }
input[type="file"] { display: none; }
button {
background: #333;
@@ -57,13 +68,24 @@
margin-top: 10px;
}
button:hover { background: #555; }
button:disabled { background: #999; cursor: not-allowed; }
:focus-visible {
outline: 3px solid #0066cc;
outline-offset: 2px;
}
.progress-container {
width: 100%;
display: none;
align-items: center;
gap: 10px;
margin: 10px 0;
}
.progress-container.active { display: flex; }
.progress-track {
flex: 1;
background: #ddd;
border-radius: 4px;
margin: 10px 0;
height: 20px;
display: none;
overflow: hidden;
}
.progress-bar {
height: 100%;
@@ -72,6 +94,7 @@
width: 0;
transition: width 0.3s;
}
.progress-text { font-size: 0.85em; min-width: 4ch; text-align: right; }
.results {
margin-top: 20px;
}
@@ -82,17 +105,43 @@
border-radius: 4px;
word-break: break-all;
}
.result-item a {
.result-item.error {
background: #fdecea;
color: #8a1f11;
border-left: 4px solid #c0392b;
}
.result-name {
font-weight: bold;
margin-bottom: 4px;
}
.result-row {
display: flex;
align-items: flex-start;
gap: 10px;
}
.result-row a {
color: #0066cc;
flex: 1;
word-break: break-all;
}
.copy-button {
margin-top: 0;
padding: 4px 12px;
font-size: 0.85em;
flex-shrink: 0;
}
.version {
position: fixed;
left: 8px;
bottom: 6px;
color: #999;
color: #767676;
font-size: 0.75em;
}
@media (prefers-reduced-motion: reduce) {
* { transition: none !important; }
}
@media (max-height: 800px) {
body { margin: 15px auto; padding: 15px; }
h1 { margin-bottom: 15px; font-size: 1.6em; }
@@ -110,37 +159,44 @@
code { padding: 10px; font-size: 0.85em; }
.upload-area { padding: 20px 15px; }
button { padding: 12px 20px; }
.copy-button { padding: 8px 12px; }
.result-row { flex-direction: column; }
}
</style>
</head>
<body>
<h1>🚀 Beam - File Upload Service</h1>
<h1><span aria-hidden="true">🚀</span> Beam - File Upload Service</h1>
<div class="section">
<h2>Upload with curl</h2>
<code>curl --upload-file myfile.txt {{ .Proto }}://{{ .Host }}/myfile.txt</code>
<p style="margin-top: 10px;">The server will respond with a download URL.</p>
<p style="margin-top: 10px;">Maximum upload size: <strong id="maxUploadSizeLabel" data-max-bytes="{{ .MaxUploadSize }}">{{ .MaxUploadSize }} bytes</strong></p>
<p class="note">The server responds with the download URL.</p>
<p class="note">Maximum upload size: <strong id="maxUploadSizeLabel" data-max-bytes="{{ .MaxUploadSize }}">{{ .MaxUploadSize }} bytes</strong>.
Uploaded files are deleted after <strong>{{ .Expiry }}</strong>.</p>
</div>
<div class="section">
<h2>Download a file</h2>
<code>curl {{ .Proto }}://{{ .Host }}/x123abcx/myfile.txt -o myfile.txt</code>
<p style="margin-top: 10px;">Or simply open the URL in your browser.</p>
<p class="note">Or simply open the URL in your browser.</p>
</div>
<div class="section">
<h2>Web Upload</h2>
<div class="upload-area" id="uploadArea">
<p>📁 Click to select files or drag and drop here</p>
<p style="margin-top: 10px; font-size: 0.85em; color: #666;">Maximum file size: <span id="maxUploadSizeText"></span></p>
<div class="upload-area" id="uploadArea" role="button" tabindex="0" aria-label="Select files to upload, or drag and drop them here">
<p><span aria-hidden="true">📁</span> Click to select files or drag and drop here</p>
<p class="hint">Maximum total size: <span id="maxUploadSizeText"></span> &middot; deleted after {{ .Expiry }}</p>
<input type="file" id="fileInput" multiple>
<button type="button" onclick="event.stopPropagation(); document.getElementById('fileInput').click()">Select Files</button>
<button type="button" id="selectButton">Select Files</button>
<button type="button" id="cancelButton" hidden>Cancel</button>
</div>
<div class="progress-container" id="progressContainer">
<div class="progress-bar" id="progressBar"></div>
<div class="progress-track">
<div class="progress-bar" id="progressBar" role="progressbar" aria-valuemin="0" aria-valuemax="100" aria-valuenow="0" aria-label="Upload progress"></div>
</div>
<div class="results" id="results"></div>
<span class="progress-text" id="progressText">0%</span>
</div>
<div class="results" id="results" role="status" aria-live="polite"></div>
</div>
<div class="version">v{{ .Version }}</div>
@@ -148,11 +204,18 @@
<script>
const uploadArea = document.getElementById('uploadArea');
const fileInput = document.getElementById('fileInput');
const selectButton = document.getElementById('selectButton');
const cancelButton = document.getElementById('cancelButton');
const progressContainer = document.getElementById('progressContainer');
const progressBar = document.getElementById('progressBar');
const progressText = document.getElementById('progressText');
const results = document.getElementById('results');
const maxUploadSize = parseInt(document.getElementById('maxUploadSizeLabel').dataset.maxBytes, 10) || 0;
let currentXhr = null;
let uploading = false;
let dragDepth = 0;
function formatBytes(bytes) {
if (!bytes) return 'unknown';
const units = ['B', 'KB', 'MB', 'GB', 'TB'];
@@ -168,43 +231,200 @@
document.getElementById('maxUploadSizeLabel').textContent = formatBytes(maxUploadSize);
document.getElementById('maxUploadSizeText').textContent = formatBytes(maxUploadSize);
function clearResults() {
results.replaceChildren();
function setProgress(percent) {
const rounded = Math.round(percent);
progressBar.style.width = percent + '%';
progressBar.setAttribute('aria-valuenow', rounded);
progressText.textContent = rounded + '%';
}
function showMessage(message) {
function showProgress() {
progressContainer.classList.add('active');
setProgress(0);
}
// Hide rather than leave a stale bar behind: a failed upload must not
// display a full, complete-looking bar.
function hideProgress() {
progressContainer.classList.remove('active');
setProgress(0);
}
function setUploading(active) {
uploading = active;
selectButton.disabled = active;
cancelButton.hidden = !active;
uploadArea.classList.toggle('disabled', active);
uploadArea.setAttribute('aria-disabled', String(active));
}
function showMessage(message, isError) {
const div = document.createElement('div');
div.className = 'result-item';
div.className = isError ? 'result-item error' : 'result-item';
div.textContent = message;
results.replaceChildren(div);
}
function filenameFromURL(url) {
try {
const path = new URL(url, window.location.href).pathname;
return decodeURIComponent(path.substring(path.lastIndexOf('/') + 1));
} catch (e) {
return url;
}
}
async function copyToClipboard(text) {
// The async Clipboard API requires a secure context, so it is
// unavailable when beam is reached over plain HTTP.
if (navigator.clipboard && window.isSecureContext) {
try {
await navigator.clipboard.writeText(text);
return true;
} catch (e) { /* fall through to the legacy path */ }
}
const scratch = document.createElement('textarea');
scratch.value = text;
scratch.setAttribute('readonly', '');
scratch.style.position = 'fixed';
scratch.style.opacity = '0';
document.body.appendChild(scratch);
scratch.select();
let copied = false;
try {
copied = document.execCommand('copy');
} catch (e) {
copied = false;
}
document.body.removeChild(scratch);
return copied;
}
function resultRow(url) {
const div = document.createElement('div');
div.className = 'result-item';
const name = document.createElement('div');
name.className = 'result-name';
name.textContent = filenameFromURL(url);
div.appendChild(name);
const row = document.createElement('div');
row.className = 'result-row';
const link = document.createElement('a');
link.href = url;
link.target = '_blank';
link.rel = 'noopener noreferrer';
link.textContent = url;
row.appendChild(link);
const copyButton = document.createElement('button');
copyButton.type = 'button';
copyButton.className = 'copy-button';
copyButton.textContent = 'Copy';
copyButton.setAttribute('aria-label', 'Copy download URL for ' + filenameFromURL(url));
copyButton.addEventListener('click', async () => {
const copied = await copyToClipboard(url);
copyButton.textContent = copied ? 'Copied!' : 'Press Ctrl+C';
if (!copied) {
const range = document.createRange();
range.selectNodeContents(link);
const selection = window.getSelection();
selection.removeAllRanges();
selection.addRange(range);
}
setTimeout(() => { copyButton.textContent = 'Copy'; }, 1500);
});
row.appendChild(copyButton);
div.appendChild(row);
return div;
}
function displayResults(urls, failed) {
if (!Array.isArray(urls)) {
showMessage('Upload failed: Unexpected response from server', true);
return;
}
const fragment = document.createDocumentFragment();
const heading = document.createElement('h3');
heading.textContent = urls.length === 1 ? '1 file uploaded' : urls.length + ' files uploaded';
fragment.appendChild(heading);
urls.forEach(url => fragment.appendChild(resultRow(url)));
(failed || []).forEach(name => {
const div = document.createElement('div');
div.className = 'result-item error';
div.textContent = 'Failed to upload: ' + name;
fragment.appendChild(div);
});
results.replaceChildren(fragment);
}
selectButton.addEventListener('click', (e) => {
e.stopPropagation();
fileInput.click();
});
cancelButton.addEventListener('click', (e) => {
e.stopPropagation();
if (currentXhr) currentXhr.abort();
});
uploadArea.addEventListener('click', (e) => {
if (e.target.closest('button')) return;
if (uploading) return;
fileInput.click();
});
uploadArea.addEventListener('keydown', (e) => {
if (e.target.closest('button')) return;
if (e.key !== 'Enter' && e.key !== ' ') return;
e.preventDefault();
if (!uploading) fileInput.click();
});
// Track enter/leave depth: without it, dragging over the child
// elements fires dragleave and the highlight flickers.
uploadArea.addEventListener('dragenter', (e) => {
e.preventDefault();
dragDepth++;
if (!uploading) uploadArea.classList.add('dragover');
});
uploadArea.addEventListener('dragover', (e) => {
e.preventDefault();
uploadArea.classList.add('dragover');
});
uploadArea.addEventListener('dragleave', () => {
uploadArea.classList.remove('dragover');
dragDepth = Math.max(0, dragDepth - 1);
if (dragDepth === 0) uploadArea.classList.remove('dragover');
});
uploadArea.addEventListener('drop', (e) => {
e.preventDefault();
dragDepth = 0;
uploadArea.classList.remove('dragover');
const files = e.dataTransfer.files;
uploadFiles(files);
uploadFiles(e.dataTransfer.files);
});
fileInput.addEventListener('change', (e) => {
uploadFiles(e.target.files);
// Reset so re-selecting the same file fires change again.
fileInput.value = '';
});
function uploadFiles(files) {
if (uploading) {
showMessage('An upload is already in progress.', true);
return;
}
if (files.length === 0) return;
const oversizedFiles = [];
@@ -218,13 +438,15 @@
if (oversizedFiles.length > 0) {
showMessage('Upload failed: the following file(s) exceed the maximum size of ' +
formatBytes(maxUploadSize) + ': ' + oversizedFiles.join(', '));
formatBytes(maxUploadSize) + ': ' + oversizedFiles.join(', '), true);
return;
}
// The server limits the whole request body, so the total matters
// even when each individual file fits.
if (maxUploadSize && totalSize > maxUploadSize) {
showMessage('Upload failed: total upload size (' + formatBytes(totalSize) +
') exceeds the maximum of ' + formatBytes(maxUploadSize));
') exceeds the maximum of ' + formatBytes(maxUploadSize), true);
return;
}
@@ -233,27 +455,29 @@
formData.append('files', file);
}
progressContainer.style.display = 'block';
progressBar.style.width = '0%';
clearResults();
showProgress();
results.replaceChildren();
setUploading(true);
const xhr = new XMLHttpRequest();
currentXhr = xhr;
xhr.upload.addEventListener('progress', (e) => {
if (e.lengthComputable) {
const percent = (e.loaded / e.total) * 100;
progressBar.style.width = percent + '%';
setProgress((e.loaded / e.total) * 100);
}
});
xhr.addEventListener('load', () => {
if (xhr.status === 200) {
let response;
try {
const response = JSON.parse(xhr.responseText);
displayResults(response.urls);
response = JSON.parse(xhr.responseText);
} catch (e) {
showMessage('Upload failed: Unexpected response from server');
showMessage('Upload failed: Unexpected response from server', true);
return;
}
displayResults(response.urls, response.failed);
} else {
let message = xhr.statusText;
try {
@@ -266,37 +490,30 @@
message = xhr.responseText;
}
}
showMessage('Upload failed: ' + message);
showMessage('Upload failed: ' + message, true);
}
progressBar.style.width = '100%';
});
xhr.addEventListener('error', () => {
showMessage('Upload failed: Network error');
showMessage('Upload failed: Network error', true);
});
xhr.addEventListener('abort', () => {
showMessage('Upload cancelled.', true);
});
// Runs after load/error/abort, so every outcome clears the bar.
// No xhr.timeout: it caps the whole request and would kill
// legitimate multi-gigabyte uploads.
xhr.addEventListener('loadend', () => {
currentXhr = null;
setUploading(false);
hideProgress();
});
xhr.open('POST', '/upload');
xhr.send(formData);
}
function displayResults(urls) {
const heading = document.createElement('h3');
heading.textContent = 'Download URLs:';
results.replaceChildren(heading);
urls.forEach(url => {
const div = document.createElement('div');
div.className = 'result-item';
const link = document.createElement('a');
link.href = url;
link.target = '_blank';
link.rel = 'noopener noreferrer';
link.textContent = url;
div.appendChild(link);
results.appendChild(div);
});
}
</script>
</body>
</html>
main.go
+409 -136
View File
@@ -3,6 +3,7 @@ package main
import (
"context"
"crypto/rand"
_ "embed"
"encoding/json"
"errors"
"fmt"
@@ -11,7 +12,9 @@ import (
"io/fs"
"log"
"mime"
"mime/multipart"
"net/http"
"net/url"
"os"
"os/signal"
"path/filepath"
@@ -34,8 +37,33 @@ const defaultMaxUploadSize int64 = 5 << 30 // 5GB
// maxUploadSize.
const maxMultipartMemory int64 = 32 << 20 // 32MB
var indexHTML *template.Template
// maxFilesPerUpload bounds how many parts a single multipart upload may
// contain, so one request cannot exhaust inodes with tiny files.
const maxFilesPerUpload = 100
// defaultShutdownGrace is used when SHUTDOWN_GRACE is not set. Align it with
// the pod's terminationGracePeriodSeconds when uploads are expected to be
// large and slow.
const defaultShutdownGrace = 15 * time.Second
// defaultCleanupInterval is used when CLEANUP_INTERVAL is not set.
const defaultCleanupInterval = 6 * time.Hour
// defaultFileExpires is used when FILE_DURATION is not set.
const defaultFileExpires = 7 * 24 * time.Hour
// newUploadGrace keeps the cleanup sweep from removing an upload directory
// that was just created but whose file has not been written yet.
const newUploadGrace = 5 * time.Minute
// Embedded so the binary does not depend on its working directory.
//
//go:embed index.html.tmpl
var indexTemplate string
var indexHTML = template.Must(template.New("index").Parse(indexTemplate))
var maxUploadSize int64 = defaultMaxUploadSize
var fileExpires = defaultFileExpires
var version = "dev"
func main() {
@@ -43,26 +71,66 @@ func main() {
if uploadDir == "" {
uploadDir = "/uploads"
}
var fileExpires time.Duration = 7 * 24 * time.Hour
// DRYRUN_CLEAN defaults to true: cleanup reports what it would delete but
// removes nothing until it is explicitly disabled.
dryRun := true
if dryRunEnv := os.Getenv("DRYRUN_CLEAN"); dryRunEnv != "" {
parsed, err := strconv.ParseBool(dryRunEnv)
if err != nil {
log.Fatalf("DRYRUN_CLEAN %s is invalid: %v", dryRunEnv, err)
}
dryRun = parsed
}
dryRun := strings.ToLower(os.Getenv("DRYRUN_CLEAN")) != "false"
fileDuration := os.Getenv("FILE_DURATION")
if fileDuration != "" {
expires, err := time.ParseDuration(fileDuration)
if err != nil {
log.Fatalf("FILE_DURATION %s is invalid: %v", fileDuration, err)
}
// A negative duration would make every file look expired on the very
// first sweep and wipe the upload directory.
if expires < 0 {
log.Fatalf("FILE_DURATION %s is invalid: must not be negative", fileDuration)
}
fileExpires = expires
}
if maxUploadSizeEnv := os.Getenv("MAX_UPLOAD_SIZE"); maxUploadSizeEnv != "" {
size, err := strconv.ParseInt(maxUploadSizeEnv, 10, 64)
if err != nil || size <= 0 {
if err != nil {
log.Fatalf("MAX_UPLOAD_SIZE %s is invalid: %v", maxUploadSizeEnv, err)
}
if size <= 0 {
log.Fatalf("MAX_UPLOAD_SIZE %s is invalid: must be positive", maxUploadSizeEnv)
}
maxUploadSize = size
}
shutdownGrace := defaultShutdownGrace
if graceEnv := os.Getenv("SHUTDOWN_GRACE"); graceEnv != "" {
grace, err := time.ParseDuration(graceEnv)
if err != nil {
log.Fatalf("SHUTDOWN_GRACE %s is invalid: %v", graceEnv, err)
}
if grace <= 0 {
log.Fatalf("SHUTDOWN_GRACE %s is invalid: must be positive", graceEnv)
}
shutdownGrace = grace
}
cleanupInterval := defaultCleanupInterval
if intervalEnv := os.Getenv("CLEANUP_INTERVAL"); intervalEnv != "" {
interval, err := time.ParseDuration(intervalEnv)
if err != nil {
log.Fatalf("CLEANUP_INTERVAL %s is invalid: %v", intervalEnv, err)
}
if interval <= 0 {
log.Fatalf("CLEANUP_INTERVAL %s is invalid: must be positive", intervalEnv)
}
cleanupInterval = interval
}
port := os.Getenv("PORT")
if port == "" {
port = "8080"
@@ -71,15 +139,20 @@ func main() {
if err := os.MkdirAll(uploadDir, 0755); err != nil {
log.Fatalf("Failed to create upload directory: %v", err)
}
indexHTML = template.Must(template.ParseFiles("index.html.tmpl"))
http.HandleFunc("/", handleRequest)
mux := http.NewServeMux()
mux.HandleFunc("/", handleRequest)
log.Printf("Starting server on port %s, upload directory: %s", port, uploadDir)
log.Printf("Version: %s", version)
log.Printf("Files expires after: %v (DRYRUN_CLEAN=%v)", fileExpires, dryRun)
log.Printf("Max upload size: %d bytes", maxUploadSize)
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
if fileExpires != 0 {
go cleanupProcedure(fileExpires, dryRun)
log.Printf("Cleanup interval: %v", cleanupInterval)
go cleanupProcedure(ctx, fileExpires, cleanupInterval, dryRun)
} else {
log.Print("Cleaning old files is disabled")
}
@@ -90,69 +163,65 @@ func main() {
// style attacks on connection/header handling.
srv := &http.Server{
Addr: ":" + port,
Handler: withLogging(mux),
ReadHeaderTimeout: 10 * time.Second,
IdleTimeout: 120 * time.Second,
}
serverErr := make(chan error, 1)
go func() {
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
serverErr <- err
}
close(serverErr)
}()
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
select {
case err := <-serverErr:
log.Fatalf("Server failed: %v", err)
case <-ctx.Done():
log.Print("Shutdown signal received, shutting down gracefully...")
shutdownCtx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
shutdownCtx, cancel := context.WithTimeout(context.Background(), shutdownGrace)
defer cancel()
// A large upload in flight will not finish inside the grace period.
// Force the remaining connections closed and still exit cleanly, so a
// rollout mid-transfer does not look like a crash to Kubernetes.
if err := srv.Shutdown(shutdownCtx); err != nil {
log.Fatalf("Graceful shutdown failed: %v", err)
log.Printf("Graceful shutdown did not complete within %v (%v), forcing close", shutdownGrace, err)
srv.Close()
}
log.Print("Server stopped")
}
}
func handleRequest(w http.ResponseWriter, r *http.Request) {
start := time.Now()
if r.Method == "GET" && r.URL.Path == "/" {
if r.Method == http.MethodGet && r.URL.Path == "/" {
serveFrontpage(w, r)
logRequest(r, 200, time.Since(start))
return
}
if r.Method == "GET" && r.URL.Path == "/healthz" {
if r.Method == http.MethodGet && r.URL.Path == "/healthz" {
handleHealthz(w, r)
return
}
if r.Method == "POST" && r.URL.Path == "/upload" {
handleMultipartUpload(w, r, start)
if r.Method == http.MethodPost && r.URL.Path == "/upload" {
handleMultipartUpload(w, r)
return
}
if r.Method == "PUT" {
handleUpload(w, r, start)
if r.Method == http.MethodPut {
handleUpload(w, r)
return
}
if r.Method == "GET" {
handleDownload(w, r, start)
// ServeContent handles HEAD correctly, so downloads answer both verbs.
if r.Method == http.MethodGet || r.Method == http.MethodHead {
handleDownload(w, r)
return
}
if r.Method == "HEAD" {
return
}
w.WriteHeader(http.StatusMethodNotAllowed)
logRequest(r, 405, time.Since(start))
w.Header().Set("Allow", "GET, HEAD, PUT, POST")
http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
}
func handleHealthz(w http.ResponseWriter, r *http.Request) {
@@ -164,20 +233,23 @@ func handleHealthz(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
}
func handleUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
filename := strings.TrimPrefix(r.URL.Path, "/")
if filename == "" {
func handleUpload(w http.ResponseWriter, r *http.Request) {
rawName := strings.TrimPrefix(r.URL.Path, "/")
if rawName == "" {
http.Error(w, "Filename required", http.StatusBadRequest)
logRequest(r, 400, time.Since(start))
return
}
filename = filepath.Base(filename)
filename, ok := safeFilename(rawName)
if !ok {
http.Error(w, "Invalid filename", http.StatusBadRequest)
return
}
id, err := generateUniqueID()
if err != nil {
log.Printf("Failed to generate unique ID: %v", err)
http.Error(w, "Failed to generate unique ID", http.StatusInternalServerError)
logRequest(r, 500, time.Since(start))
return
}
@@ -186,13 +258,15 @@ func handleUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
file, err := os.Create(targetPath)
if err != nil {
os.Remove(targetDir)
log.Printf("Failed to create %q: %v", targetPath, err)
http.Error(w, "Failed to create file", http.StatusInternalServerError)
logRequest(r, 500, time.Since(start))
return
}
defer file.Close()
limitedBody := http.MaxBytesReader(w, r.Body, maxUploadSize)
// MaxBytesReader needs the ResponseWriter the stdlib handed us so it can
// mark the connection for close; the logging wrapper would hide it.
limitedBody := http.MaxBytesReader(baseWriter(w), r.Body, maxUploadSize)
written, err := io.Copy(file, limitedBody)
if err != nil {
file.Close()
@@ -200,23 +274,39 @@ func handleUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
os.Remove(targetDir)
if isMaxBytesError(err) {
http.Error(w, fmt.Sprintf("File exceeds maximum upload size of %d bytes", maxUploadSize), http.StatusRequestEntityTooLarge)
logRequest(r, 413, time.Since(start))
return
}
log.Printf("Failed to write %q: %v", targetPath, err)
http.Error(w, "Failed to write file", http.StatusInternalServerError)
logRequest(r, 500, time.Since(start))
return
}
proto := getProtocol(r)
host := getHost(r)
downloadURL := fmt.Sprintf("%s://%s/%s/%s\n", proto, host, id, filename)
link := downloadURL(r, id, filename)
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.WriteHeader(http.StatusOK)
w.Write([]byte(downloadURL))
w.Write([]byte(link + "\n"))
log.Printf("Uploaded %s (%d bytes) -> %s", filename, written, downloadURL)
logRequest(r, 200, time.Since(start))
log.Printf("Uploaded %q (%d bytes) -> %s", filename, written, link)
}
// safeFilename reduces an untrusted name to its base component and reports
// whether the result is safe to join onto the upload directory. Note that
// filepath.IsLocal alone is not enough: it accepts ".", and filepath.Base
// turns an empty name into "." as well.
func safeFilename(name string) (string, bool) {
base := filepath.Base(name)
if base == "." || base == ".." || !filepath.IsLocal(base) {
return "", false
}
return base, true
}
// downloadURL builds the externally visible URL for a stored file. The
// filename is percent-escaped so that spaces, '#' and '?' survive the round
// trip back through handleDownload.
func downloadURL(r *http.Request, id, filename string) string {
return fmt.Sprintf("%s://%s/%s/%s", getProtocol(r), getHost(r), id, url.PathEscape(filename))
}
func writeMultipartError(w http.ResponseWriter, status int, message string) {
@@ -225,94 +315,130 @@ func writeMultipartError(w http.ResponseWriter, status int, message string) {
json.NewEncoder(w).Encode(map[string]string{"error": message})
}
func handleMultipartUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
r.Body = http.MaxBytesReader(w, r.Body, maxUploadSize)
// uploadResponse is the success body for POST /upload. Failed carries the
// names of parts that could not be stored, so a partial failure is visible to
// the client instead of silently yielding a short urls list.
type uploadResponse struct {
URLs []string `json:"urls"`
Failed []string `json:"failed,omitempty"`
}
func handleMultipartUpload(w http.ResponseWriter, r *http.Request) {
// See handleUpload: MaxBytesReader must see the original ResponseWriter.
r.Body = http.MaxBytesReader(baseWriter(w), r.Body, maxUploadSize)
memoryLimit := maxUploadSize
if memoryLimit > maxMultipartMemory {
memoryLimit = maxMultipartMemory
}
// Registered before ParseMultipartForm: a parse failure can still leave
// spilled temporary files behind, and the error paths below return early.
defer func() {
if r.MultipartForm != nil {
r.MultipartForm.RemoveAll()
}
}()
if err := r.ParseMultipartForm(memoryLimit); err != nil {
if isMaxBytesError(err) {
writeMultipartError(w, http.StatusRequestEntityTooLarge, fmt.Sprintf("Upload exceeds maximum size of %d bytes", maxUploadSize))
logRequest(r, 413, time.Since(start))
return
}
writeMultipartError(w, http.StatusBadRequest, "Failed to parse multipart form")
logRequest(r, 400, time.Since(start))
return
}
defer r.MultipartForm.RemoveAll()
files := r.MultipartForm.File["files"]
if len(files) == 0 {
writeMultipartError(w, http.StatusBadRequest, "No files provided")
logRequest(r, 400, time.Since(start))
return
}
if len(files) > maxFilesPerUpload {
writeMultipartError(w, http.StatusBadRequest, fmt.Sprintf("Too many files: %d, maximum is %d", len(files), maxFilesPerUpload))
return
}
proto := getProtocol(r)
host := getHost(r)
urls := make([]string, 0, len(files))
resp := uploadResponse{URLs: make([]string, 0, len(files))}
for _, fileHeader := range files {
src, err := fileHeader.Open()
if err != nil {
filename, ok := safeFilename(fileHeader.Filename)
if !ok {
log.Printf("Rejected upload part with invalid filename %q", fileHeader.Filename)
resp.Failed = append(resp.Failed, fileHeader.Filename)
continue
}
link, err := storeUploadedFile(r, fileHeader, filename)
if err != nil {
log.Printf("Failed to store uploaded file %q: %v", filename, err)
resp.Failed = append(resp.Failed, filename)
continue
}
resp.URLs = append(resp.URLs, link)
}
// Every part failed: this is a server-side problem, not a 200.
if len(resp.URLs) == 0 {
writeMultipartError(w, http.StatusInternalServerError, "Failed to store uploaded files")
return
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(resp)
}
// storeUploadedFile writes one multipart part to its own upload directory and
// returns the download URL. Any partially written file or directory is removed
// before returning an error.
func storeUploadedFile(r *http.Request, fileHeader *multipart.FileHeader, filename string) (string, error) {
src, err := fileHeader.Open()
if err != nil {
return "", err
}
defer src.Close()
id, err := generateUniqueID()
if err != nil {
src.Close()
continue
return "", err
}
filename := filepath.Base(fileHeader.Filename)
targetDir := filepath.Join(uploadDir, id)
targetPath := filepath.Join(targetDir, filename)
dst, err := os.Create(targetPath)
if err != nil {
src.Close()
os.Remove(targetDir)
continue
return "", err
}
written, err := io.Copy(dst, src)
dst.Close()
src.Close()
closeErr := dst.Close()
if err == nil {
err = closeErr
}
if err != nil {
os.Remove(targetPath)
os.Remove(targetDir)
continue
return "", err
}
downloadURL := fmt.Sprintf("%s://%s/%s/%s", proto, host, id, filename)
urls = append(urls, downloadURL)
log.Printf("Uploaded %s (%d bytes) -> %s", filename, written, downloadURL)
link := downloadURL(r, id, filename)
log.Printf("Uploaded %q (%d bytes) -> %s", filename, written, link)
return link, nil
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(map[string][]string{"urls": urls})
logRequest(r, 200, time.Since(start))
}
func handleDownload(w http.ResponseWriter, r *http.Request, start time.Time) {
func handleDownload(w http.ResponseWriter, r *http.Request) {
parts := strings.Split(strings.TrimPrefix(r.URL.Path, "/"), "/")
if len(parts) != 2 {
http.NotFound(w, r)
logRequest(r, 404, time.Since(start))
return
}
id := parts[0]
filename := parts[1]
if !filepath.IsLocal(id) || !filepath.IsLocal(filename) {
id, idOK := safeFilename(parts[0])
filename, nameOK := safeFilename(parts[1])
if !idOK || !nameOK {
http.NotFound(w, r)
logRequest(r, 404, time.Since(start))
return
}
@@ -320,24 +446,37 @@ func handleDownload(w http.ResponseWriter, r *http.Request, start time.Time) {
file, err := os.Open(filePath)
if err != nil {
http.NotFound(w, r)
logRequest(r, 404, time.Since(start))
return
}
defer file.Close()
stat, err := file.Stat()
if err != nil {
log.Printf("Failed to stat %q: %v", filePath, err)
http.Error(w, "Failed to stat file", http.StatusInternalServerError)
logRequest(r, 500, time.Since(start))
return
}
// A directory would otherwise be served as a zero-length file.
if stat.IsDir() {
http.NotFound(w, r)
return
}
// Set Content-Type explicitly so ServeContent does not sniff the body.
// The extension lookup relies on the system MIME database, which the
// Dockerfile provides via the mailcap package.
w.Header().Set("Content-Type", detectContentType(filename))
w.Header().Set("Content-Disposition", mime.FormatMediaType("inline", map[string]string{"filename": filename}))
w.Header().Set("Content-Length", fmt.Sprintf("%d", stat.Size()))
// FormatMediaType returns "" for names it cannot encode; don't emit an
// empty header in that case.
if disposition := mime.FormatMediaType("inline", map[string]string{"filename": filename}); disposition != "" {
w.Header().Set("Content-Disposition", disposition)
}
// Download URLs are unguessable secrets, so keep shared caches out of it.
w.Header().Set("Cache-Control", "private")
io.Copy(w, file)
logRequest(r, 200, time.Since(start))
// ServeContent handles Range, If-Modified-Since, Content-Length and HEAD,
// which makes interrupted downloads resumable with `curl -C -`.
http.ServeContent(w, r, filename, stat.ModTime(), file)
}
func generateUniqueID() (string, error) {
@@ -437,22 +576,105 @@ func detectContentType(filename string) string {
return "application/octet-stream"
}
func logRequest(r *http.Request, status int, duration time.Duration) {
log.Printf("[%s] %s %s %d %v", time.Now().Format("2006-01-02 15:04:05"), r.Method, r.URL.Path, status, duration)
// responseRecorder captures the status code and byte count actually written,
// so the access log reports what happened rather than what the handler
// intended. http.ServeContent in particular may answer 206 or 304.
type responseRecorder struct {
http.ResponseWriter
status int
written int64
}
func (rr *responseRecorder) WriteHeader(status int) {
if rr.status == 0 {
rr.status = status
}
rr.ResponseWriter.WriteHeader(status)
}
func (rr *responseRecorder) Write(b []byte) (int, error) {
if rr.status == 0 {
rr.status = http.StatusOK
}
n, err := rr.ResponseWriter.Write(b)
rr.written += int64(n)
return n, err
}
// Unwrap lets http.ResponseController and baseWriter reach the original
// ResponseWriter.
func (rr *responseRecorder) Unwrap() http.ResponseWriter { return rr.ResponseWriter }
// baseWriter returns the ResponseWriter the net/http server created. It is
// needed by http.MaxBytesReader, which type-asserts its argument to an
// unexported interface to mark an oversized request's connection for close;
// that assertion fails against a wrapper.
func baseWriter(w http.ResponseWriter) http.ResponseWriter {
for {
u, ok := w.(interface{ Unwrap() http.ResponseWriter })
if !ok {
return w
}
w = u.Unwrap()
}
}
// withLogging records one access-log line per request, including for handlers
// that never write an explicit status.
func withLogging(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
rec := &responseRecorder{ResponseWriter: w}
next.ServeHTTP(rec, r)
status := rec.status
if status == 0 {
status = http.StatusOK
}
// %q on the path: it is attacker-controlled and would otherwise let a
// request forge log lines by embedding newlines.
log.Printf("%s %q %d %d %v", r.Method, r.URL.Path, status, rec.written, time.Since(start))
})
}
func serveFrontpage(w http.ResponseWriter, r *http.Request) {
proto := getProtocol(r)
host := getHost(r)
pageData := map[string]any{
"Proto": proto,
"Host": host,
"Proto": getProtocol(r),
"Host": getHost(r),
"MaxUploadSize": maxUploadSize,
"Expiry": formatExpiry(fileExpires),
"Version": version,
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
indexHTML.Execute(w, pageData)
if err := indexHTML.Execute(w, pageData); err != nil {
log.Printf("Failed to render front page: %v", err)
}
}
// formatExpiry renders a retention duration for the front page.
// Duration.String() would produce "168h0m0s" for the default week.
func formatExpiry(d time.Duration) string {
switch {
case d <= 0:
return "never"
case d%(24*time.Hour) == 0:
return plural(int(d/(24*time.Hour)), "day")
case d%time.Hour == 0:
return plural(int(d/time.Hour), "hour")
case d%time.Minute == 0:
return plural(int(d/time.Minute), "minute")
default:
return d.String()
}
}
func plural(n int, unit string) string {
if n == 1 {
return "1 " + unit
}
return fmt.Sprintf("%d %ss", n, unit)
}
// Cleanup
@@ -476,12 +698,51 @@ func isDirEmpty(name string) (bool, error) {
return false, err
}
func cleanupProcedure(duration time.Duration, dryRun bool) {
var oldFiles []string
var emptyDirs []string
func cleanupProcedure(ctx context.Context, duration, interval time.Duration, dryRun bool) {
ticker := time.NewTicker(interval)
defer ticker.Stop()
deleteIfTooOld := func(path string, d fs.DirEntry, err error) error {
for {
cleanupCycle(duration, dryRun)
select {
case <-ctx.Done():
log.Print("Cleanup stopped")
return
case <-ticker.C:
}
}
}
// cleanupCycle performs one sweep: expired files first, then the upload
// directories left empty behind them.
func cleanupCycle(duration time.Duration, dryRun bool) {
oldFiles, err := collectExpiredFiles(uploadDir, duration)
if err != nil {
log.Printf("Cleanup: failed to scan %q for expired files: %v", uploadDir, err)
}
removedFiles := removeAll(oldFiles, "expired file", dryRun)
emptyDirs, err := collectEmptyDirs(uploadDir)
if err != nil {
log.Printf("Cleanup: failed to scan %q for empty folders: %v", uploadDir, err)
}
removedDirs := removeAll(emptyDirs, "empty folder", dryRun)
log.Printf("Cleanup finished: %d expired file(s) and %d empty folder(s) found, %d and %d removed (dryRun=%v)",
len(oldFiles), len(emptyDirs), removedFiles, removedDirs, dryRun)
}
// collectExpiredFiles returns files under root whose mtime is at least
// duration old.
func collectExpiredFiles(root string, duration time.Duration) ([]string, error) {
var oldFiles []string
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
if err != nil {
log.Printf("Cleanup: skipping %q: %v", path, err)
return nil
}
if d.IsDir() {
@@ -490,64 +751,76 @@ func cleanupProcedure(duration time.Duration, dryRun bool) {
info, err := d.Info()
if err != nil {
log.Printf("Error cleaning file '%s' : %v", d.Name(), err)
log.Printf("Cleanup: failed to stat %q: %v", path, err)
return nil
}
age := time.Since(info.ModTime())
if age >= duration {
if time.Since(info.ModTime()) >= duration {
oldFiles = append(oldFiles, path)
}
return nil
})
return oldFiles, err
}
findEmptyDirs := func(path string, d fs.DirEntry, err error) error {
// collectEmptyDirs returns empty directories under root, excluding root itself
// and any directory created within newUploadGrace, which may belong to an
// upload that has not written its file yet.
func collectEmptyDirs(root string) ([]string, error) {
var emptyDirs []string
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
// d is nil when the root itself cannot be read, so this check must
// come first.
if err != nil {
log.Printf("Cleanup: skipping %q: %v", path, err)
return nil
}
if !d.IsDir() {
return nil
}
if filepath.Clean(path) == filepath.Clean(uploadDir) {
if filepath.Clean(path) == filepath.Clean(root) {
return nil
}
info, err := d.Info()
if err != nil {
log.Printf("Cleanup: failed to stat %q: %v", path, err)
return nil
}
if time.Since(info.ModTime()) < newUploadGrace {
return nil
}
empty, err := isDirEmpty(path)
if err != nil {
log.Printf("Cleanup: failed to read %q: %v", path, err)
return nil
}
if empty {
emptyDirs = append(emptyDirs, path)
} else if err != nil {
log.Printf("Error cleaning directory '%s' : %v", path, err)
}
return nil
})
return emptyDirs, err
}
for {
oldFiles = oldFiles[:0]
emptyDirs = emptyDirs[:0]
filepath.WalkDir(uploadDir, deleteIfTooOld)
for _, file := range oldFiles {
if !dryRun {
err := os.Remove(file)
if err != nil {
log.Printf("Failed to remove expired file '%s' : %v", file, err)
} else {
log.Printf("Removed expired file '%s'", file)
// removeAll deletes the given paths unless dryRun is set, and returns how many
// were actually removed.
func removeAll(paths []string, kind string, dryRun bool) int {
removed := 0
for _, path := range paths {
if dryRun {
log.Printf("Dry run: Would remove %s %q", kind, path)
continue
}
} else {
log.Printf("Dry run: Would remove expired file '%s'", file)
if err := os.Remove(path); err != nil {
log.Printf("Failed to remove %s %q: %v", kind, path, err)
continue
}
log.Printf("Removed %s %q", kind, path)
removed++
}
filepath.WalkDir(uploadDir, findEmptyDirs)
for _, file := range emptyDirs {
if !dryRun {
err := os.Remove(file)
if err != nil {
log.Printf("Failed to remove empty folder '%s' : %v", file, err)
} else {
log.Printf("Removed empty folder '%s'", file)
}
} else {
log.Printf("Dry run: Would remove empty folder '%s'", file)
}
}
time.Sleep(6 * time.Hour)
}
return removed
}