Add Improve UI error handling, add download resuming, improve cleanup
code
This commit is contained in:
@@ -4,3 +4,10 @@ uploads/
|
||||
.gitignore
|
||||
Dockerfile
|
||||
.dockerignore
|
||||
beam
|
||||
*_test.go
|
||||
.vscode/
|
||||
.github/
|
||||
.clinerules
|
||||
CLAUDE.md
|
||||
.claude/
|
||||
|
||||
@@ -4,3 +4,4 @@ beam
|
||||
*.dll
|
||||
*.so
|
||||
*.dylib
|
||||
.vscode/
|
||||
|
||||
+1
-1
@@ -27,7 +27,7 @@ RUN apk add --no-cache mailcap \
|
||||
USER beam
|
||||
WORKDIR /app
|
||||
|
||||
COPY --from=builder /build/index.html.tmpl /build/beam .
|
||||
COPY --from=builder /build/beam .
|
||||
|
||||
EXPOSE 8080
|
||||
|
||||
|
||||
@@ -1,10 +1,130 @@
|
||||
# beam
|
||||
|
||||
File transfer service via http.
|
||||
File transfer service via http. Upload with `curl` or a browser, share the
|
||||
returned URL, and the file is deleted automatically once it expires.
|
||||
|
||||
## Usage
|
||||
|
||||
Upload a single file with `PUT`; the response body is the download URL:
|
||||
|
||||
```sh
|
||||
curl --upload-file myfile.txt https://beam.example.com/myfile.txt
|
||||
# https://beam.example.com/x123abcx/myfile.txt
|
||||
```
|
||||
|
||||
Upload one or more files as a multipart form. This is the endpoint the web UI
|
||||
uses, and it responds with JSON:
|
||||
|
||||
```sh
|
||||
curl -F 'files=@a.txt' -F 'files=@b.txt' https://beam.example.com/upload
|
||||
# {"urls":["https://beam.example.com/x123abcx/a.txt", "..."]}
|
||||
```
|
||||
|
||||
If some parts could not be stored, their names are returned alongside the
|
||||
successful URLs:
|
||||
|
||||
```json
|
||||
{"urls": ["https://beam.example.com/x123abcx/a.txt"], "failed": ["b.txt"]}
|
||||
```
|
||||
|
||||
Errors use a consistent shape:
|
||||
|
||||
```json
|
||||
{"error": "Upload exceeds maximum size of 5368709120 bytes"}
|
||||
```
|
||||
|
||||
Download a file, or just open the URL in a browser:
|
||||
|
||||
```sh
|
||||
curl https://beam.example.com/x123abcx/myfile.txt -o myfile.txt
|
||||
```
|
||||
|
||||
Downloads support range requests, so an interrupted transfer can be resumed:
|
||||
|
||||
```sh
|
||||
curl -C - -O https://beam.example.com/x123abcx/myfile.txt
|
||||
```
|
||||
|
||||
### Endpoints
|
||||
|
||||
| Method | Path | Purpose |
|
||||
| ------------ | -------------------- | ---------------------------------------- |
|
||||
| `GET` | `/` | Web UI with usage instructions |
|
||||
| `PUT` | `/{filename}` | Upload one file, returns the URL as text |
|
||||
| `POST` | `/upload` | Multipart upload, returns JSON |
|
||||
| `GET`/`HEAD` | `/{id}/{filename}` | Download, supports `Range` |
|
||||
| `GET` | `/healthz` | Liveness/readiness probe |
|
||||
|
||||
## Configuration
|
||||
|
||||
Configuration is via environment variables only. Invalid values are rejected at
|
||||
startup.
|
||||
|
||||
| Variable | Default | Description |
|
||||
| ------------------ | ---------- | ----------------------------------------------------------------------- |
|
||||
| `UPLOAD_DIR` | `/uploads` | Where uploaded files are stored. |
|
||||
| `PORT` | `8080` | Port to listen on. |
|
||||
| `FILE_DURATION` | `168h` | How long a file is kept. Set to `0` to disable cleanup entirely. |
|
||||
| `DRYRUN_CLEAN` | `true` | When true, cleanup only logs what it *would* delete. See the note below. |
|
||||
| `CLEANUP_INTERVAL` | `6h` | How often the cleanup sweep runs. |
|
||||
| `MAX_UPLOAD_SIZE` | `5368709120` | Maximum bytes per request (5 GiB), covering all files in one upload. |
|
||||
| `SHUTDOWN_GRACE` | `15s` | How long to wait for in-flight requests on SIGTERM. |
|
||||
|
||||
> **`DRYRUN_CLEAN` defaults to `true`.** Until you set `DRYRUN_CLEAN=false`,
|
||||
> expired files are never actually deleted — the service only logs
|
||||
> `Dry run: Would remove expired file ...` while the disk keeps filling up.
|
||||
|
||||
Durations use Go's `time.ParseDuration` syntax (`30m`, `12h`, `168h`).
|
||||
`MAX_UPLOAD_SIZE` is a plain byte count.
|
||||
|
||||
Notes on behaviour worth knowing:
|
||||
|
||||
- `MAX_UPLOAD_SIZE` limits the **whole request**, so a multipart upload of
|
||||
several files is measured by their combined size, not per file.
|
||||
- A single multipart request may contain at most 100 files.
|
||||
- Expiry is based on the file's modification time; downloading does not extend
|
||||
a file's life.
|
||||
- `SHUTDOWN_GRACE` should be aligned with the pod's
|
||||
`terminationGracePeriodSeconds`. If an upload is still running when the grace
|
||||
period elapses, remaining connections are closed and the process still exits
|
||||
cleanly.
|
||||
|
||||
## Running
|
||||
|
||||
```sh
|
||||
docker run -p 8080:8080 \
|
||||
-e DRYRUN_CLEAN=false \
|
||||
-e FILE_DURATION=168h \
|
||||
-v /srv/beam-uploads:/uploads \
|
||||
docker.sdal.tech/beam:latest
|
||||
```
|
||||
|
||||
The container runs as the non-root user `beam` (uid 1000), which must be able
|
||||
to write to whatever is mounted at `/uploads`.
|
||||
|
||||
## Development
|
||||
|
||||
```sh
|
||||
go build ./... && go vet ./... && gofmt -l . && go test ./...
|
||||
```
|
||||
|
||||
Run locally:
|
||||
|
||||
```sh
|
||||
UPLOAD_DIR=/tmp/beam-uploads PORT=8080 DRYRUN_CLEAN=false go run .
|
||||
```
|
||||
|
||||
Note that without a reverse proxy setting `X-Forwarded-Host`, generated
|
||||
download URLs fall back to `localhost:8080`. In production beam runs behind a
|
||||
TLS-terminating ingress that sets `X-Forwarded-Proto` and `X-Forwarded-Host`,
|
||||
which is where the URLs come from.
|
||||
|
||||
## Versioning
|
||||
|
||||
The Dockerfile owns the base application version with the `VERSION` build argument. Docker builds append the current git short hash with `GIT_SHA` so the homepage can display the exact image version near the bottom-left edge.
|
||||
`build.sh` owns the application version via the `VERSION` variable and passes
|
||||
it to the Docker build as a build argument. Builds append the current git short
|
||||
hash with `GIT_SHA` so the homepage can display the exact image version near the
|
||||
bottom-left edge.
|
||||
|
||||
```sh
|
||||
./build.sh
|
||||
|
||||
+273
-56
@@ -6,15 +6,19 @@
|
||||
<title>Beam - File Upload Service</title>
|
||||
<style>
|
||||
* { margin: 0; padding: 0; box-sizing: border-box; }
|
||||
html { color-scheme: light; }
|
||||
body {
|
||||
font-family: monospace;
|
||||
max-width: 800px;
|
||||
margin: 40px auto;
|
||||
padding: 20px;
|
||||
line-height: 1.6;
|
||||
background: #fff;
|
||||
color: #222;
|
||||
}
|
||||
h1 { margin-bottom: 30px; font-size: 2em; }
|
||||
h2 { margin: 30px 0 15px 0; font-size: 1.3em; }
|
||||
h3 { margin: 0 0 10px 0; font-size: 1.05em; }
|
||||
.section {
|
||||
background: #f5f5f5;
|
||||
padding: 20px;
|
||||
@@ -30,12 +34,13 @@
|
||||
border-radius: 4px;
|
||||
overflow-x: auto;
|
||||
}
|
||||
.note { margin-top: 10px; }
|
||||
.upload-area {
|
||||
border: 3px dashed #ccc;
|
||||
padding: 40px;
|
||||
text-align: center;
|
||||
cursor: pointer;
|
||||
transition: all 0.3s;
|
||||
transition: border-color 0.3s, background 0.3s;
|
||||
}
|
||||
.upload-area.dragover {
|
||||
border-color: #333;
|
||||
@@ -44,6 +49,12 @@
|
||||
.upload-area:hover {
|
||||
border-color: #666;
|
||||
}
|
||||
.upload-area.disabled {
|
||||
cursor: not-allowed;
|
||||
opacity: 0.6;
|
||||
}
|
||||
.upload-area.disabled:hover { border-color: #ccc; }
|
||||
.hint { margin-top: 10px; font-size: 0.85em; color: #666; }
|
||||
input[type="file"] { display: none; }
|
||||
button {
|
||||
background: #333;
|
||||
@@ -57,13 +68,24 @@
|
||||
margin-top: 10px;
|
||||
}
|
||||
button:hover { background: #555; }
|
||||
button:disabled { background: #999; cursor: not-allowed; }
|
||||
:focus-visible {
|
||||
outline: 3px solid #0066cc;
|
||||
outline-offset: 2px;
|
||||
}
|
||||
.progress-container {
|
||||
width: 100%;
|
||||
display: none;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
margin: 10px 0;
|
||||
}
|
||||
.progress-container.active { display: flex; }
|
||||
.progress-track {
|
||||
flex: 1;
|
||||
background: #ddd;
|
||||
border-radius: 4px;
|
||||
margin: 10px 0;
|
||||
height: 20px;
|
||||
display: none;
|
||||
overflow: hidden;
|
||||
}
|
||||
.progress-bar {
|
||||
height: 100%;
|
||||
@@ -72,6 +94,7 @@
|
||||
width: 0;
|
||||
transition: width 0.3s;
|
||||
}
|
||||
.progress-text { font-size: 0.85em; min-width: 4ch; text-align: right; }
|
||||
.results {
|
||||
margin-top: 20px;
|
||||
}
|
||||
@@ -82,17 +105,43 @@
|
||||
border-radius: 4px;
|
||||
word-break: break-all;
|
||||
}
|
||||
.result-item a {
|
||||
.result-item.error {
|
||||
background: #fdecea;
|
||||
color: #8a1f11;
|
||||
border-left: 4px solid #c0392b;
|
||||
}
|
||||
.result-name {
|
||||
font-weight: bold;
|
||||
margin-bottom: 4px;
|
||||
}
|
||||
.result-row {
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
gap: 10px;
|
||||
}
|
||||
.result-row a {
|
||||
color: #0066cc;
|
||||
flex: 1;
|
||||
word-break: break-all;
|
||||
}
|
||||
.copy-button {
|
||||
margin-top: 0;
|
||||
padding: 4px 12px;
|
||||
font-size: 0.85em;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.version {
|
||||
position: fixed;
|
||||
left: 8px;
|
||||
bottom: 6px;
|
||||
color: #999;
|
||||
color: #767676;
|
||||
font-size: 0.75em;
|
||||
}
|
||||
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
* { transition: none !important; }
|
||||
}
|
||||
|
||||
@media (max-height: 800px) {
|
||||
body { margin: 15px auto; padding: 15px; }
|
||||
h1 { margin-bottom: 15px; font-size: 1.6em; }
|
||||
@@ -110,37 +159,44 @@
|
||||
code { padding: 10px; font-size: 0.85em; }
|
||||
.upload-area { padding: 20px 15px; }
|
||||
button { padding: 12px 20px; }
|
||||
.copy-button { padding: 8px 12px; }
|
||||
.result-row { flex-direction: column; }
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1>🚀 Beam - File Upload Service</h1>
|
||||
<h1><span aria-hidden="true">🚀</span> Beam - File Upload Service</h1>
|
||||
|
||||
<div class="section">
|
||||
<h2>Upload with curl</h2>
|
||||
<code>curl --upload-file myfile.txt {{ .Proto }}://{{ .Host }}/myfile.txt</code>
|
||||
<p style="margin-top: 10px;">The server will respond with a download URL.</p>
|
||||
<p style="margin-top: 10px;">Maximum upload size: <strong id="maxUploadSizeLabel" data-max-bytes="{{ .MaxUploadSize }}">{{ .MaxUploadSize }} bytes</strong></p>
|
||||
<p class="note">The server responds with the download URL.</p>
|
||||
<p class="note">Maximum upload size: <strong id="maxUploadSizeLabel" data-max-bytes="{{ .MaxUploadSize }}">{{ .MaxUploadSize }} bytes</strong>.
|
||||
Uploaded files are deleted after <strong>{{ .Expiry }}</strong>.</p>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<h2>Download a file</h2>
|
||||
<code>curl {{ .Proto }}://{{ .Host }}/x123abcx/myfile.txt -o myfile.txt</code>
|
||||
<p style="margin-top: 10px;">Or simply open the URL in your browser.</p>
|
||||
<p class="note">Or simply open the URL in your browser.</p>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
<h2>Web Upload</h2>
|
||||
<div class="upload-area" id="uploadArea">
|
||||
<p>📁 Click to select files or drag and drop here</p>
|
||||
<p style="margin-top: 10px; font-size: 0.85em; color: #666;">Maximum file size: <span id="maxUploadSizeText"></span></p>
|
||||
<div class="upload-area" id="uploadArea" role="button" tabindex="0" aria-label="Select files to upload, or drag and drop them here">
|
||||
<p><span aria-hidden="true">📁</span> Click to select files or drag and drop here</p>
|
||||
<p class="hint">Maximum total size: <span id="maxUploadSizeText"></span> · deleted after {{ .Expiry }}</p>
|
||||
<input type="file" id="fileInput" multiple>
|
||||
<button type="button" onclick="event.stopPropagation(); document.getElementById('fileInput').click()">Select Files</button>
|
||||
<button type="button" id="selectButton">Select Files</button>
|
||||
<button type="button" id="cancelButton" hidden>Cancel</button>
|
||||
</div>
|
||||
<div class="progress-container" id="progressContainer">
|
||||
<div class="progress-bar" id="progressBar"></div>
|
||||
<div class="progress-track">
|
||||
<div class="progress-bar" id="progressBar" role="progressbar" aria-valuemin="0" aria-valuemax="100" aria-valuenow="0" aria-label="Upload progress"></div>
|
||||
</div>
|
||||
<div class="results" id="results"></div>
|
||||
<span class="progress-text" id="progressText">0%</span>
|
||||
</div>
|
||||
<div class="results" id="results" role="status" aria-live="polite"></div>
|
||||
</div>
|
||||
|
||||
<div class="version">v{{ .Version }}</div>
|
||||
@@ -148,11 +204,18 @@
|
||||
<script>
|
||||
const uploadArea = document.getElementById('uploadArea');
|
||||
const fileInput = document.getElementById('fileInput');
|
||||
const selectButton = document.getElementById('selectButton');
|
||||
const cancelButton = document.getElementById('cancelButton');
|
||||
const progressContainer = document.getElementById('progressContainer');
|
||||
const progressBar = document.getElementById('progressBar');
|
||||
const progressText = document.getElementById('progressText');
|
||||
const results = document.getElementById('results');
|
||||
const maxUploadSize = parseInt(document.getElementById('maxUploadSizeLabel').dataset.maxBytes, 10) || 0;
|
||||
|
||||
let currentXhr = null;
|
||||
let uploading = false;
|
||||
let dragDepth = 0;
|
||||
|
||||
function formatBytes(bytes) {
|
||||
if (!bytes) return 'unknown';
|
||||
const units = ['B', 'KB', 'MB', 'GB', 'TB'];
|
||||
@@ -168,43 +231,200 @@
|
||||
document.getElementById('maxUploadSizeLabel').textContent = formatBytes(maxUploadSize);
|
||||
document.getElementById('maxUploadSizeText').textContent = formatBytes(maxUploadSize);
|
||||
|
||||
function clearResults() {
|
||||
results.replaceChildren();
|
||||
function setProgress(percent) {
|
||||
const rounded = Math.round(percent);
|
||||
progressBar.style.width = percent + '%';
|
||||
progressBar.setAttribute('aria-valuenow', rounded);
|
||||
progressText.textContent = rounded + '%';
|
||||
}
|
||||
|
||||
function showMessage(message) {
|
||||
function showProgress() {
|
||||
progressContainer.classList.add('active');
|
||||
setProgress(0);
|
||||
}
|
||||
|
||||
// Hide rather than leave a stale bar behind: a failed upload must not
|
||||
// display a full, complete-looking bar.
|
||||
function hideProgress() {
|
||||
progressContainer.classList.remove('active');
|
||||
setProgress(0);
|
||||
}
|
||||
|
||||
function setUploading(active) {
|
||||
uploading = active;
|
||||
selectButton.disabled = active;
|
||||
cancelButton.hidden = !active;
|
||||
uploadArea.classList.toggle('disabled', active);
|
||||
uploadArea.setAttribute('aria-disabled', String(active));
|
||||
}
|
||||
|
||||
function showMessage(message, isError) {
|
||||
const div = document.createElement('div');
|
||||
div.className = 'result-item';
|
||||
div.className = isError ? 'result-item error' : 'result-item';
|
||||
div.textContent = message;
|
||||
results.replaceChildren(div);
|
||||
}
|
||||
|
||||
function filenameFromURL(url) {
|
||||
try {
|
||||
const path = new URL(url, window.location.href).pathname;
|
||||
return decodeURIComponent(path.substring(path.lastIndexOf('/') + 1));
|
||||
} catch (e) {
|
||||
return url;
|
||||
}
|
||||
}
|
||||
|
||||
async function copyToClipboard(text) {
|
||||
// The async Clipboard API requires a secure context, so it is
|
||||
// unavailable when beam is reached over plain HTTP.
|
||||
if (navigator.clipboard && window.isSecureContext) {
|
||||
try {
|
||||
await navigator.clipboard.writeText(text);
|
||||
return true;
|
||||
} catch (e) { /* fall through to the legacy path */ }
|
||||
}
|
||||
|
||||
const scratch = document.createElement('textarea');
|
||||
scratch.value = text;
|
||||
scratch.setAttribute('readonly', '');
|
||||
scratch.style.position = 'fixed';
|
||||
scratch.style.opacity = '0';
|
||||
document.body.appendChild(scratch);
|
||||
scratch.select();
|
||||
let copied = false;
|
||||
try {
|
||||
copied = document.execCommand('copy');
|
||||
} catch (e) {
|
||||
copied = false;
|
||||
}
|
||||
document.body.removeChild(scratch);
|
||||
return copied;
|
||||
}
|
||||
|
||||
function resultRow(url) {
|
||||
const div = document.createElement('div');
|
||||
div.className = 'result-item';
|
||||
|
||||
const name = document.createElement('div');
|
||||
name.className = 'result-name';
|
||||
name.textContent = filenameFromURL(url);
|
||||
div.appendChild(name);
|
||||
|
||||
const row = document.createElement('div');
|
||||
row.className = 'result-row';
|
||||
|
||||
const link = document.createElement('a');
|
||||
link.href = url;
|
||||
link.target = '_blank';
|
||||
link.rel = 'noopener noreferrer';
|
||||
link.textContent = url;
|
||||
row.appendChild(link);
|
||||
|
||||
const copyButton = document.createElement('button');
|
||||
copyButton.type = 'button';
|
||||
copyButton.className = 'copy-button';
|
||||
copyButton.textContent = 'Copy';
|
||||
copyButton.setAttribute('aria-label', 'Copy download URL for ' + filenameFromURL(url));
|
||||
copyButton.addEventListener('click', async () => {
|
||||
const copied = await copyToClipboard(url);
|
||||
copyButton.textContent = copied ? 'Copied!' : 'Press Ctrl+C';
|
||||
if (!copied) {
|
||||
const range = document.createRange();
|
||||
range.selectNodeContents(link);
|
||||
const selection = window.getSelection();
|
||||
selection.removeAllRanges();
|
||||
selection.addRange(range);
|
||||
}
|
||||
setTimeout(() => { copyButton.textContent = 'Copy'; }, 1500);
|
||||
});
|
||||
row.appendChild(copyButton);
|
||||
|
||||
div.appendChild(row);
|
||||
return div;
|
||||
}
|
||||
|
||||
function displayResults(urls, failed) {
|
||||
if (!Array.isArray(urls)) {
|
||||
showMessage('Upload failed: Unexpected response from server', true);
|
||||
return;
|
||||
}
|
||||
|
||||
const fragment = document.createDocumentFragment();
|
||||
|
||||
const heading = document.createElement('h3');
|
||||
heading.textContent = urls.length === 1 ? '1 file uploaded' : urls.length + ' files uploaded';
|
||||
fragment.appendChild(heading);
|
||||
|
||||
urls.forEach(url => fragment.appendChild(resultRow(url)));
|
||||
|
||||
(failed || []).forEach(name => {
|
||||
const div = document.createElement('div');
|
||||
div.className = 'result-item error';
|
||||
div.textContent = 'Failed to upload: ' + name;
|
||||
fragment.appendChild(div);
|
||||
});
|
||||
|
||||
results.replaceChildren(fragment);
|
||||
}
|
||||
|
||||
selectButton.addEventListener('click', (e) => {
|
||||
e.stopPropagation();
|
||||
fileInput.click();
|
||||
});
|
||||
|
||||
cancelButton.addEventListener('click', (e) => {
|
||||
e.stopPropagation();
|
||||
if (currentXhr) currentXhr.abort();
|
||||
});
|
||||
|
||||
uploadArea.addEventListener('click', (e) => {
|
||||
if (e.target.closest('button')) return;
|
||||
if (uploading) return;
|
||||
fileInput.click();
|
||||
});
|
||||
|
||||
uploadArea.addEventListener('keydown', (e) => {
|
||||
if (e.target.closest('button')) return;
|
||||
if (e.key !== 'Enter' && e.key !== ' ') return;
|
||||
e.preventDefault();
|
||||
if (!uploading) fileInput.click();
|
||||
});
|
||||
|
||||
// Track enter/leave depth: without it, dragging over the child
|
||||
// elements fires dragleave and the highlight flickers.
|
||||
uploadArea.addEventListener('dragenter', (e) => {
|
||||
e.preventDefault();
|
||||
dragDepth++;
|
||||
if (!uploading) uploadArea.classList.add('dragover');
|
||||
});
|
||||
|
||||
uploadArea.addEventListener('dragover', (e) => {
|
||||
e.preventDefault();
|
||||
uploadArea.classList.add('dragover');
|
||||
});
|
||||
|
||||
uploadArea.addEventListener('dragleave', () => {
|
||||
uploadArea.classList.remove('dragover');
|
||||
dragDepth = Math.max(0, dragDepth - 1);
|
||||
if (dragDepth === 0) uploadArea.classList.remove('dragover');
|
||||
});
|
||||
|
||||
uploadArea.addEventListener('drop', (e) => {
|
||||
e.preventDefault();
|
||||
dragDepth = 0;
|
||||
uploadArea.classList.remove('dragover');
|
||||
const files = e.dataTransfer.files;
|
||||
uploadFiles(files);
|
||||
uploadFiles(e.dataTransfer.files);
|
||||
});
|
||||
|
||||
fileInput.addEventListener('change', (e) => {
|
||||
uploadFiles(e.target.files);
|
||||
// Reset so re-selecting the same file fires change again.
|
||||
fileInput.value = '';
|
||||
});
|
||||
|
||||
function uploadFiles(files) {
|
||||
if (uploading) {
|
||||
showMessage('An upload is already in progress.', true);
|
||||
return;
|
||||
}
|
||||
if (files.length === 0) return;
|
||||
|
||||
const oversizedFiles = [];
|
||||
@@ -218,13 +438,15 @@
|
||||
|
||||
if (oversizedFiles.length > 0) {
|
||||
showMessage('Upload failed: the following file(s) exceed the maximum size of ' +
|
||||
formatBytes(maxUploadSize) + ': ' + oversizedFiles.join(', '));
|
||||
formatBytes(maxUploadSize) + ': ' + oversizedFiles.join(', '), true);
|
||||
return;
|
||||
}
|
||||
|
||||
// The server limits the whole request body, so the total matters
|
||||
// even when each individual file fits.
|
||||
if (maxUploadSize && totalSize > maxUploadSize) {
|
||||
showMessage('Upload failed: total upload size (' + formatBytes(totalSize) +
|
||||
') exceeds the maximum of ' + formatBytes(maxUploadSize));
|
||||
') exceeds the maximum of ' + formatBytes(maxUploadSize), true);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -233,27 +455,29 @@
|
||||
formData.append('files', file);
|
||||
}
|
||||
|
||||
progressContainer.style.display = 'block';
|
||||
progressBar.style.width = '0%';
|
||||
clearResults();
|
||||
showProgress();
|
||||
results.replaceChildren();
|
||||
setUploading(true);
|
||||
|
||||
const xhr = new XMLHttpRequest();
|
||||
currentXhr = xhr;
|
||||
|
||||
xhr.upload.addEventListener('progress', (e) => {
|
||||
if (e.lengthComputable) {
|
||||
const percent = (e.loaded / e.total) * 100;
|
||||
progressBar.style.width = percent + '%';
|
||||
setProgress((e.loaded / e.total) * 100);
|
||||
}
|
||||
});
|
||||
|
||||
xhr.addEventListener('load', () => {
|
||||
if (xhr.status === 200) {
|
||||
let response;
|
||||
try {
|
||||
const response = JSON.parse(xhr.responseText);
|
||||
displayResults(response.urls);
|
||||
response = JSON.parse(xhr.responseText);
|
||||
} catch (e) {
|
||||
showMessage('Upload failed: Unexpected response from server');
|
||||
showMessage('Upload failed: Unexpected response from server', true);
|
||||
return;
|
||||
}
|
||||
displayResults(response.urls, response.failed);
|
||||
} else {
|
||||
let message = xhr.statusText;
|
||||
try {
|
||||
@@ -266,37 +490,30 @@
|
||||
message = xhr.responseText;
|
||||
}
|
||||
}
|
||||
showMessage('Upload failed: ' + message);
|
||||
showMessage('Upload failed: ' + message, true);
|
||||
}
|
||||
progressBar.style.width = '100%';
|
||||
});
|
||||
|
||||
xhr.addEventListener('error', () => {
|
||||
showMessage('Upload failed: Network error');
|
||||
showMessage('Upload failed: Network error', true);
|
||||
});
|
||||
|
||||
xhr.addEventListener('abort', () => {
|
||||
showMessage('Upload cancelled.', true);
|
||||
});
|
||||
|
||||
// Runs after load/error/abort, so every outcome clears the bar.
|
||||
// No xhr.timeout: it caps the whole request and would kill
|
||||
// legitimate multi-gigabyte uploads.
|
||||
xhr.addEventListener('loadend', () => {
|
||||
currentXhr = null;
|
||||
setUploading(false);
|
||||
hideProgress();
|
||||
});
|
||||
|
||||
xhr.open('POST', '/upload');
|
||||
xhr.send(formData);
|
||||
}
|
||||
|
||||
function displayResults(urls) {
|
||||
const heading = document.createElement('h3');
|
||||
heading.textContent = 'Download URLs:';
|
||||
results.replaceChildren(heading);
|
||||
|
||||
urls.forEach(url => {
|
||||
const div = document.createElement('div');
|
||||
div.className = 'result-item';
|
||||
const link = document.createElement('a');
|
||||
link.href = url;
|
||||
link.target = '_blank';
|
||||
link.rel = 'noopener noreferrer';
|
||||
link.textContent = url;
|
||||
div.appendChild(link);
|
||||
results.appendChild(div);
|
||||
});
|
||||
}
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
|
||||
@@ -3,6 +3,7 @@ package main
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
_ "embed"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
@@ -11,7 +12,9 @@ import (
|
||||
"io/fs"
|
||||
"log"
|
||||
"mime"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"os/signal"
|
||||
"path/filepath"
|
||||
@@ -34,8 +37,33 @@ const defaultMaxUploadSize int64 = 5 << 30 // 5GB
|
||||
// maxUploadSize.
|
||||
const maxMultipartMemory int64 = 32 << 20 // 32MB
|
||||
|
||||
var indexHTML *template.Template
|
||||
// maxFilesPerUpload bounds how many parts a single multipart upload may
|
||||
// contain, so one request cannot exhaust inodes with tiny files.
|
||||
const maxFilesPerUpload = 100
|
||||
|
||||
// defaultShutdownGrace is used when SHUTDOWN_GRACE is not set. Align it with
|
||||
// the pod's terminationGracePeriodSeconds when uploads are expected to be
|
||||
// large and slow.
|
||||
const defaultShutdownGrace = 15 * time.Second
|
||||
|
||||
// defaultCleanupInterval is used when CLEANUP_INTERVAL is not set.
|
||||
const defaultCleanupInterval = 6 * time.Hour
|
||||
|
||||
// defaultFileExpires is used when FILE_DURATION is not set.
|
||||
const defaultFileExpires = 7 * 24 * time.Hour
|
||||
|
||||
// newUploadGrace keeps the cleanup sweep from removing an upload directory
|
||||
// that was just created but whose file has not been written yet.
|
||||
const newUploadGrace = 5 * time.Minute
|
||||
|
||||
// Embedded so the binary does not depend on its working directory.
|
||||
//
|
||||
//go:embed index.html.tmpl
|
||||
var indexTemplate string
|
||||
|
||||
var indexHTML = template.Must(template.New("index").Parse(indexTemplate))
|
||||
var maxUploadSize int64 = defaultMaxUploadSize
|
||||
var fileExpires = defaultFileExpires
|
||||
var version = "dev"
|
||||
|
||||
func main() {
|
||||
@@ -43,26 +71,66 @@ func main() {
|
||||
if uploadDir == "" {
|
||||
uploadDir = "/uploads"
|
||||
}
|
||||
var fileExpires time.Duration = 7 * 24 * time.Hour
|
||||
// DRYRUN_CLEAN defaults to true: cleanup reports what it would delete but
|
||||
// removes nothing until it is explicitly disabled.
|
||||
dryRun := true
|
||||
if dryRunEnv := os.Getenv("DRYRUN_CLEAN"); dryRunEnv != "" {
|
||||
parsed, err := strconv.ParseBool(dryRunEnv)
|
||||
if err != nil {
|
||||
log.Fatalf("DRYRUN_CLEAN %s is invalid: %v", dryRunEnv, err)
|
||||
}
|
||||
dryRun = parsed
|
||||
}
|
||||
|
||||
dryRun := strings.ToLower(os.Getenv("DRYRUN_CLEAN")) != "false"
|
||||
fileDuration := os.Getenv("FILE_DURATION")
|
||||
if fileDuration != "" {
|
||||
expires, err := time.ParseDuration(fileDuration)
|
||||
if err != nil {
|
||||
log.Fatalf("FILE_DURATION %s is invalid: %v", fileDuration, err)
|
||||
}
|
||||
// A negative duration would make every file look expired on the very
|
||||
// first sweep and wipe the upload directory.
|
||||
if expires < 0 {
|
||||
log.Fatalf("FILE_DURATION %s is invalid: must not be negative", fileDuration)
|
||||
}
|
||||
fileExpires = expires
|
||||
}
|
||||
|
||||
if maxUploadSizeEnv := os.Getenv("MAX_UPLOAD_SIZE"); maxUploadSizeEnv != "" {
|
||||
size, err := strconv.ParseInt(maxUploadSizeEnv, 10, 64)
|
||||
if err != nil || size <= 0 {
|
||||
if err != nil {
|
||||
log.Fatalf("MAX_UPLOAD_SIZE %s is invalid: %v", maxUploadSizeEnv, err)
|
||||
}
|
||||
if size <= 0 {
|
||||
log.Fatalf("MAX_UPLOAD_SIZE %s is invalid: must be positive", maxUploadSizeEnv)
|
||||
}
|
||||
maxUploadSize = size
|
||||
}
|
||||
|
||||
shutdownGrace := defaultShutdownGrace
|
||||
if graceEnv := os.Getenv("SHUTDOWN_GRACE"); graceEnv != "" {
|
||||
grace, err := time.ParseDuration(graceEnv)
|
||||
if err != nil {
|
||||
log.Fatalf("SHUTDOWN_GRACE %s is invalid: %v", graceEnv, err)
|
||||
}
|
||||
if grace <= 0 {
|
||||
log.Fatalf("SHUTDOWN_GRACE %s is invalid: must be positive", graceEnv)
|
||||
}
|
||||
shutdownGrace = grace
|
||||
}
|
||||
|
||||
cleanupInterval := defaultCleanupInterval
|
||||
if intervalEnv := os.Getenv("CLEANUP_INTERVAL"); intervalEnv != "" {
|
||||
interval, err := time.ParseDuration(intervalEnv)
|
||||
if err != nil {
|
||||
log.Fatalf("CLEANUP_INTERVAL %s is invalid: %v", intervalEnv, err)
|
||||
}
|
||||
if interval <= 0 {
|
||||
log.Fatalf("CLEANUP_INTERVAL %s is invalid: must be positive", intervalEnv)
|
||||
}
|
||||
cleanupInterval = interval
|
||||
}
|
||||
|
||||
port := os.Getenv("PORT")
|
||||
if port == "" {
|
||||
port = "8080"
|
||||
@@ -71,15 +139,20 @@ func main() {
|
||||
if err := os.MkdirAll(uploadDir, 0755); err != nil {
|
||||
log.Fatalf("Failed to create upload directory: %v", err)
|
||||
}
|
||||
indexHTML = template.Must(template.ParseFiles("index.html.tmpl"))
|
||||
http.HandleFunc("/", handleRequest)
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/", handleRequest)
|
||||
|
||||
log.Printf("Starting server on port %s, upload directory: %s", port, uploadDir)
|
||||
log.Printf("Version: %s", version)
|
||||
log.Printf("Files expires after: %v (DRYRUN_CLEAN=%v)", fileExpires, dryRun)
|
||||
log.Printf("Max upload size: %d bytes", maxUploadSize)
|
||||
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
|
||||
if fileExpires != 0 {
|
||||
go cleanupProcedure(fileExpires, dryRun)
|
||||
log.Printf("Cleanup interval: %v", cleanupInterval)
|
||||
go cleanupProcedure(ctx, fileExpires, cleanupInterval, dryRun)
|
||||
} else {
|
||||
log.Print("Cleaning old files is disabled")
|
||||
}
|
||||
@@ -90,69 +163,65 @@ func main() {
|
||||
// style attacks on connection/header handling.
|
||||
srv := &http.Server{
|
||||
Addr: ":" + port,
|
||||
Handler: withLogging(mux),
|
||||
ReadHeaderTimeout: 10 * time.Second,
|
||||
IdleTimeout: 120 * time.Second,
|
||||
}
|
||||
|
||||
serverErr := make(chan error, 1)
|
||||
go func() {
|
||||
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
|
||||
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
serverErr <- err
|
||||
}
|
||||
close(serverErr)
|
||||
}()
|
||||
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
|
||||
select {
|
||||
case err := <-serverErr:
|
||||
log.Fatalf("Server failed: %v", err)
|
||||
case <-ctx.Done():
|
||||
log.Print("Shutdown signal received, shutting down gracefully...")
|
||||
shutdownCtx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
|
||||
shutdownCtx, cancel := context.WithTimeout(context.Background(), shutdownGrace)
|
||||
defer cancel()
|
||||
// A large upload in flight will not finish inside the grace period.
|
||||
// Force the remaining connections closed and still exit cleanly, so a
|
||||
// rollout mid-transfer does not look like a crash to Kubernetes.
|
||||
if err := srv.Shutdown(shutdownCtx); err != nil {
|
||||
log.Fatalf("Graceful shutdown failed: %v", err)
|
||||
log.Printf("Graceful shutdown did not complete within %v (%v), forcing close", shutdownGrace, err)
|
||||
srv.Close()
|
||||
}
|
||||
log.Print("Server stopped")
|
||||
}
|
||||
}
|
||||
|
||||
func handleRequest(w http.ResponseWriter, r *http.Request) {
|
||||
start := time.Now()
|
||||
|
||||
if r.Method == "GET" && r.URL.Path == "/" {
|
||||
if r.Method == http.MethodGet && r.URL.Path == "/" {
|
||||
serveFrontpage(w, r)
|
||||
logRequest(r, 200, time.Since(start))
|
||||
return
|
||||
}
|
||||
if r.Method == "GET" && r.URL.Path == "/healthz" {
|
||||
if r.Method == http.MethodGet && r.URL.Path == "/healthz" {
|
||||
handleHealthz(w, r)
|
||||
return
|
||||
}
|
||||
|
||||
if r.Method == "POST" && r.URL.Path == "/upload" {
|
||||
handleMultipartUpload(w, r, start)
|
||||
if r.Method == http.MethodPost && r.URL.Path == "/upload" {
|
||||
handleMultipartUpload(w, r)
|
||||
return
|
||||
}
|
||||
|
||||
if r.Method == "PUT" {
|
||||
handleUpload(w, r, start)
|
||||
if r.Method == http.MethodPut {
|
||||
handleUpload(w, r)
|
||||
return
|
||||
}
|
||||
|
||||
if r.Method == "GET" {
|
||||
handleDownload(w, r, start)
|
||||
// ServeContent handles HEAD correctly, so downloads answer both verbs.
|
||||
if r.Method == http.MethodGet || r.Method == http.MethodHead {
|
||||
handleDownload(w, r)
|
||||
return
|
||||
}
|
||||
|
||||
if r.Method == "HEAD" {
|
||||
return
|
||||
}
|
||||
|
||||
w.WriteHeader(http.StatusMethodNotAllowed)
|
||||
logRequest(r, 405, time.Since(start))
|
||||
w.Header().Set("Allow", "GET, HEAD, PUT, POST")
|
||||
http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
|
||||
}
|
||||
|
||||
func handleHealthz(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -164,20 +233,23 @@ func handleHealthz(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}
|
||||
|
||||
func handleUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
|
||||
filename := strings.TrimPrefix(r.URL.Path, "/")
|
||||
if filename == "" {
|
||||
func handleUpload(w http.ResponseWriter, r *http.Request) {
|
||||
rawName := strings.TrimPrefix(r.URL.Path, "/")
|
||||
if rawName == "" {
|
||||
http.Error(w, "Filename required", http.StatusBadRequest)
|
||||
logRequest(r, 400, time.Since(start))
|
||||
return
|
||||
}
|
||||
|
||||
filename = filepath.Base(filename)
|
||||
filename, ok := safeFilename(rawName)
|
||||
if !ok {
|
||||
http.Error(w, "Invalid filename", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
id, err := generateUniqueID()
|
||||
if err != nil {
|
||||
log.Printf("Failed to generate unique ID: %v", err)
|
||||
http.Error(w, "Failed to generate unique ID", http.StatusInternalServerError)
|
||||
logRequest(r, 500, time.Since(start))
|
||||
return
|
||||
}
|
||||
|
||||
@@ -186,13 +258,15 @@ func handleUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
|
||||
file, err := os.Create(targetPath)
|
||||
if err != nil {
|
||||
os.Remove(targetDir)
|
||||
log.Printf("Failed to create %q: %v", targetPath, err)
|
||||
http.Error(w, "Failed to create file", http.StatusInternalServerError)
|
||||
logRequest(r, 500, time.Since(start))
|
||||
return
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
limitedBody := http.MaxBytesReader(w, r.Body, maxUploadSize)
|
||||
// MaxBytesReader needs the ResponseWriter the stdlib handed us so it can
|
||||
// mark the connection for close; the logging wrapper would hide it.
|
||||
limitedBody := http.MaxBytesReader(baseWriter(w), r.Body, maxUploadSize)
|
||||
written, err := io.Copy(file, limitedBody)
|
||||
if err != nil {
|
||||
file.Close()
|
||||
@@ -200,23 +274,39 @@ func handleUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
|
||||
os.Remove(targetDir)
|
||||
if isMaxBytesError(err) {
|
||||
http.Error(w, fmt.Sprintf("File exceeds maximum upload size of %d bytes", maxUploadSize), http.StatusRequestEntityTooLarge)
|
||||
logRequest(r, 413, time.Since(start))
|
||||
return
|
||||
}
|
||||
log.Printf("Failed to write %q: %v", targetPath, err)
|
||||
http.Error(w, "Failed to write file", http.StatusInternalServerError)
|
||||
logRequest(r, 500, time.Since(start))
|
||||
return
|
||||
}
|
||||
|
||||
proto := getProtocol(r)
|
||||
host := getHost(r)
|
||||
downloadURL := fmt.Sprintf("%s://%s/%s/%s\n", proto, host, id, filename)
|
||||
link := downloadURL(r, id, filename)
|
||||
|
||||
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
w.Write([]byte(downloadURL))
|
||||
w.Write([]byte(link + "\n"))
|
||||
|
||||
log.Printf("Uploaded %s (%d bytes) -> %s", filename, written, downloadURL)
|
||||
logRequest(r, 200, time.Since(start))
|
||||
log.Printf("Uploaded %q (%d bytes) -> %s", filename, written, link)
|
||||
}
|
||||
|
||||
// safeFilename reduces an untrusted name to its base component and reports
|
||||
// whether the result is safe to join onto the upload directory. Note that
|
||||
// filepath.IsLocal alone is not enough: it accepts ".", and filepath.Base
|
||||
// turns an empty name into "." as well.
|
||||
func safeFilename(name string) (string, bool) {
|
||||
base := filepath.Base(name)
|
||||
if base == "." || base == ".." || !filepath.IsLocal(base) {
|
||||
return "", false
|
||||
}
|
||||
return base, true
|
||||
}
|
||||
|
||||
// downloadURL builds the externally visible URL for a stored file. The
|
||||
// filename is percent-escaped so that spaces, '#' and '?' survive the round
|
||||
// trip back through handleDownload.
|
||||
func downloadURL(r *http.Request, id, filename string) string {
|
||||
return fmt.Sprintf("%s://%s/%s/%s", getProtocol(r), getHost(r), id, url.PathEscape(filename))
|
||||
}
|
||||
|
||||
func writeMultipartError(w http.ResponseWriter, status int, message string) {
|
||||
@@ -225,94 +315,130 @@ func writeMultipartError(w http.ResponseWriter, status int, message string) {
|
||||
json.NewEncoder(w).Encode(map[string]string{"error": message})
|
||||
}
|
||||
|
||||
func handleMultipartUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
|
||||
r.Body = http.MaxBytesReader(w, r.Body, maxUploadSize)
|
||||
// uploadResponse is the success body for POST /upload. Failed carries the
|
||||
// names of parts that could not be stored, so a partial failure is visible to
|
||||
// the client instead of silently yielding a short urls list.
|
||||
type uploadResponse struct {
|
||||
URLs []string `json:"urls"`
|
||||
Failed []string `json:"failed,omitempty"`
|
||||
}
|
||||
|
||||
func handleMultipartUpload(w http.ResponseWriter, r *http.Request) {
|
||||
// See handleUpload: MaxBytesReader must see the original ResponseWriter.
|
||||
r.Body = http.MaxBytesReader(baseWriter(w), r.Body, maxUploadSize)
|
||||
|
||||
memoryLimit := maxUploadSize
|
||||
if memoryLimit > maxMultipartMemory {
|
||||
memoryLimit = maxMultipartMemory
|
||||
}
|
||||
|
||||
// Registered before ParseMultipartForm: a parse failure can still leave
|
||||
// spilled temporary files behind, and the error paths below return early.
|
||||
defer func() {
|
||||
if r.MultipartForm != nil {
|
||||
r.MultipartForm.RemoveAll()
|
||||
}
|
||||
}()
|
||||
|
||||
if err := r.ParseMultipartForm(memoryLimit); err != nil {
|
||||
if isMaxBytesError(err) {
|
||||
writeMultipartError(w, http.StatusRequestEntityTooLarge, fmt.Sprintf("Upload exceeds maximum size of %d bytes", maxUploadSize))
|
||||
logRequest(r, 413, time.Since(start))
|
||||
return
|
||||
}
|
||||
writeMultipartError(w, http.StatusBadRequest, "Failed to parse multipart form")
|
||||
logRequest(r, 400, time.Since(start))
|
||||
return
|
||||
}
|
||||
defer r.MultipartForm.RemoveAll()
|
||||
|
||||
files := r.MultipartForm.File["files"]
|
||||
if len(files) == 0 {
|
||||
writeMultipartError(w, http.StatusBadRequest, "No files provided")
|
||||
logRequest(r, 400, time.Since(start))
|
||||
return
|
||||
}
|
||||
if len(files) > maxFilesPerUpload {
|
||||
writeMultipartError(w, http.StatusBadRequest, fmt.Sprintf("Too many files: %d, maximum is %d", len(files), maxFilesPerUpload))
|
||||
return
|
||||
}
|
||||
|
||||
proto := getProtocol(r)
|
||||
host := getHost(r)
|
||||
urls := make([]string, 0, len(files))
|
||||
resp := uploadResponse{URLs: make([]string, 0, len(files))}
|
||||
|
||||
for _, fileHeader := range files {
|
||||
src, err := fileHeader.Open()
|
||||
if err != nil {
|
||||
filename, ok := safeFilename(fileHeader.Filename)
|
||||
if !ok {
|
||||
log.Printf("Rejected upload part with invalid filename %q", fileHeader.Filename)
|
||||
resp.Failed = append(resp.Failed, fileHeader.Filename)
|
||||
continue
|
||||
}
|
||||
|
||||
link, err := storeUploadedFile(r, fileHeader, filename)
|
||||
if err != nil {
|
||||
log.Printf("Failed to store uploaded file %q: %v", filename, err)
|
||||
resp.Failed = append(resp.Failed, filename)
|
||||
continue
|
||||
}
|
||||
resp.URLs = append(resp.URLs, link)
|
||||
}
|
||||
|
||||
// Every part failed: this is a server-side problem, not a 200.
|
||||
if len(resp.URLs) == 0 {
|
||||
writeMultipartError(w, http.StatusInternalServerError, "Failed to store uploaded files")
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(resp)
|
||||
}
|
||||
|
||||
// storeUploadedFile writes one multipart part to its own upload directory and
|
||||
// returns the download URL. Any partially written file or directory is removed
|
||||
// before returning an error.
|
||||
func storeUploadedFile(r *http.Request, fileHeader *multipart.FileHeader, filename string) (string, error) {
|
||||
src, err := fileHeader.Open()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer src.Close()
|
||||
|
||||
id, err := generateUniqueID()
|
||||
if err != nil {
|
||||
src.Close()
|
||||
continue
|
||||
return "", err
|
||||
}
|
||||
|
||||
filename := filepath.Base(fileHeader.Filename)
|
||||
targetDir := filepath.Join(uploadDir, id)
|
||||
targetPath := filepath.Join(targetDir, filename)
|
||||
|
||||
dst, err := os.Create(targetPath)
|
||||
if err != nil {
|
||||
src.Close()
|
||||
os.Remove(targetDir)
|
||||
continue
|
||||
return "", err
|
||||
}
|
||||
|
||||
written, err := io.Copy(dst, src)
|
||||
dst.Close()
|
||||
src.Close()
|
||||
|
||||
closeErr := dst.Close()
|
||||
if err == nil {
|
||||
err = closeErr
|
||||
}
|
||||
if err != nil {
|
||||
os.Remove(targetPath)
|
||||
os.Remove(targetDir)
|
||||
continue
|
||||
return "", err
|
||||
}
|
||||
|
||||
downloadURL := fmt.Sprintf("%s://%s/%s/%s", proto, host, id, filename)
|
||||
urls = append(urls, downloadURL)
|
||||
log.Printf("Uploaded %s (%d bytes) -> %s", filename, written, downloadURL)
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string][]string{"urls": urls})
|
||||
logRequest(r, 200, time.Since(start))
|
||||
link := downloadURL(r, id, filename)
|
||||
log.Printf("Uploaded %q (%d bytes) -> %s", filename, written, link)
|
||||
return link, nil
|
||||
}
|
||||
|
||||
func handleDownload(w http.ResponseWriter, r *http.Request, start time.Time) {
|
||||
func handleDownload(w http.ResponseWriter, r *http.Request) {
|
||||
parts := strings.Split(strings.TrimPrefix(r.URL.Path, "/"), "/")
|
||||
if len(parts) != 2 {
|
||||
http.NotFound(w, r)
|
||||
logRequest(r, 404, time.Since(start))
|
||||
return
|
||||
}
|
||||
|
||||
id := parts[0]
|
||||
filename := parts[1]
|
||||
|
||||
if !filepath.IsLocal(id) || !filepath.IsLocal(filename) {
|
||||
id, idOK := safeFilename(parts[0])
|
||||
filename, nameOK := safeFilename(parts[1])
|
||||
if !idOK || !nameOK {
|
||||
http.NotFound(w, r)
|
||||
logRequest(r, 404, time.Since(start))
|
||||
return
|
||||
}
|
||||
|
||||
@@ -320,24 +446,37 @@ func handleDownload(w http.ResponseWriter, r *http.Request, start time.Time) {
|
||||
file, err := os.Open(filePath)
|
||||
if err != nil {
|
||||
http.NotFound(w, r)
|
||||
logRequest(r, 404, time.Since(start))
|
||||
return
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
stat, err := file.Stat()
|
||||
if err != nil {
|
||||
log.Printf("Failed to stat %q: %v", filePath, err)
|
||||
http.Error(w, "Failed to stat file", http.StatusInternalServerError)
|
||||
logRequest(r, 500, time.Since(start))
|
||||
return
|
||||
}
|
||||
// A directory would otherwise be served as a zero-length file.
|
||||
if stat.IsDir() {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
|
||||
// Set Content-Type explicitly so ServeContent does not sniff the body.
|
||||
// The extension lookup relies on the system MIME database, which the
|
||||
// Dockerfile provides via the mailcap package.
|
||||
w.Header().Set("Content-Type", detectContentType(filename))
|
||||
w.Header().Set("Content-Disposition", mime.FormatMediaType("inline", map[string]string{"filename": filename}))
|
||||
w.Header().Set("Content-Length", fmt.Sprintf("%d", stat.Size()))
|
||||
// FormatMediaType returns "" for names it cannot encode; don't emit an
|
||||
// empty header in that case.
|
||||
if disposition := mime.FormatMediaType("inline", map[string]string{"filename": filename}); disposition != "" {
|
||||
w.Header().Set("Content-Disposition", disposition)
|
||||
}
|
||||
// Download URLs are unguessable secrets, so keep shared caches out of it.
|
||||
w.Header().Set("Cache-Control", "private")
|
||||
|
||||
io.Copy(w, file)
|
||||
logRequest(r, 200, time.Since(start))
|
||||
// ServeContent handles Range, If-Modified-Since, Content-Length and HEAD,
|
||||
// which makes interrupted downloads resumable with `curl -C -`.
|
||||
http.ServeContent(w, r, filename, stat.ModTime(), file)
|
||||
}
|
||||
|
||||
func generateUniqueID() (string, error) {
|
||||
@@ -437,22 +576,105 @@ func detectContentType(filename string) string {
|
||||
return "application/octet-stream"
|
||||
}
|
||||
|
||||
func logRequest(r *http.Request, status int, duration time.Duration) {
|
||||
log.Printf("[%s] %s %s %d %v", time.Now().Format("2006-01-02 15:04:05"), r.Method, r.URL.Path, status, duration)
|
||||
// responseRecorder captures the status code and byte count actually written,
|
||||
// so the access log reports what happened rather than what the handler
|
||||
// intended. http.ServeContent in particular may answer 206 or 304.
|
||||
type responseRecorder struct {
|
||||
http.ResponseWriter
|
||||
status int
|
||||
written int64
|
||||
}
|
||||
|
||||
func (rr *responseRecorder) WriteHeader(status int) {
|
||||
if rr.status == 0 {
|
||||
rr.status = status
|
||||
}
|
||||
rr.ResponseWriter.WriteHeader(status)
|
||||
}
|
||||
|
||||
func (rr *responseRecorder) Write(b []byte) (int, error) {
|
||||
if rr.status == 0 {
|
||||
rr.status = http.StatusOK
|
||||
}
|
||||
n, err := rr.ResponseWriter.Write(b)
|
||||
rr.written += int64(n)
|
||||
return n, err
|
||||
}
|
||||
|
||||
// Unwrap lets http.ResponseController and baseWriter reach the original
|
||||
// ResponseWriter.
|
||||
func (rr *responseRecorder) Unwrap() http.ResponseWriter { return rr.ResponseWriter }
|
||||
|
||||
// baseWriter returns the ResponseWriter the net/http server created. It is
|
||||
// needed by http.MaxBytesReader, which type-asserts its argument to an
|
||||
// unexported interface to mark an oversized request's connection for close;
|
||||
// that assertion fails against a wrapper.
|
||||
func baseWriter(w http.ResponseWriter) http.ResponseWriter {
|
||||
for {
|
||||
u, ok := w.(interface{ Unwrap() http.ResponseWriter })
|
||||
if !ok {
|
||||
return w
|
||||
}
|
||||
w = u.Unwrap()
|
||||
}
|
||||
}
|
||||
|
||||
// withLogging records one access-log line per request, including for handlers
|
||||
// that never write an explicit status.
|
||||
func withLogging(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
start := time.Now()
|
||||
rec := &responseRecorder{ResponseWriter: w}
|
||||
|
||||
next.ServeHTTP(rec, r)
|
||||
|
||||
status := rec.status
|
||||
if status == 0 {
|
||||
status = http.StatusOK
|
||||
}
|
||||
// %q on the path: it is attacker-controlled and would otherwise let a
|
||||
// request forge log lines by embedding newlines.
|
||||
log.Printf("%s %q %d %d %v", r.Method, r.URL.Path, status, rec.written, time.Since(start))
|
||||
})
|
||||
}
|
||||
|
||||
func serveFrontpage(w http.ResponseWriter, r *http.Request) {
|
||||
proto := getProtocol(r)
|
||||
host := getHost(r)
|
||||
pageData := map[string]any{
|
||||
"Proto": proto,
|
||||
"Host": host,
|
||||
"Proto": getProtocol(r),
|
||||
"Host": getHost(r),
|
||||
"MaxUploadSize": maxUploadSize,
|
||||
"Expiry": formatExpiry(fileExpires),
|
||||
"Version": version,
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
indexHTML.Execute(w, pageData)
|
||||
if err := indexHTML.Execute(w, pageData); err != nil {
|
||||
log.Printf("Failed to render front page: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// formatExpiry renders a retention duration for the front page.
|
||||
// Duration.String() would produce "168h0m0s" for the default week.
|
||||
func formatExpiry(d time.Duration) string {
|
||||
switch {
|
||||
case d <= 0:
|
||||
return "never"
|
||||
case d%(24*time.Hour) == 0:
|
||||
return plural(int(d/(24*time.Hour)), "day")
|
||||
case d%time.Hour == 0:
|
||||
return plural(int(d/time.Hour), "hour")
|
||||
case d%time.Minute == 0:
|
||||
return plural(int(d/time.Minute), "minute")
|
||||
default:
|
||||
return d.String()
|
||||
}
|
||||
}
|
||||
|
||||
func plural(n int, unit string) string {
|
||||
if n == 1 {
|
||||
return "1 " + unit
|
||||
}
|
||||
return fmt.Sprintf("%d %ss", n, unit)
|
||||
}
|
||||
|
||||
// Cleanup
|
||||
@@ -476,12 +698,51 @@ func isDirEmpty(name string) (bool, error) {
|
||||
return false, err
|
||||
}
|
||||
|
||||
func cleanupProcedure(duration time.Duration, dryRun bool) {
|
||||
var oldFiles []string
|
||||
var emptyDirs []string
|
||||
func cleanupProcedure(ctx context.Context, duration, interval time.Duration, dryRun bool) {
|
||||
ticker := time.NewTicker(interval)
|
||||
defer ticker.Stop()
|
||||
|
||||
deleteIfTooOld := func(path string, d fs.DirEntry, err error) error {
|
||||
for {
|
||||
cleanupCycle(duration, dryRun)
|
||||
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
log.Print("Cleanup stopped")
|
||||
return
|
||||
case <-ticker.C:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// cleanupCycle performs one sweep: expired files first, then the upload
|
||||
// directories left empty behind them.
|
||||
func cleanupCycle(duration time.Duration, dryRun bool) {
|
||||
oldFiles, err := collectExpiredFiles(uploadDir, duration)
|
||||
if err != nil {
|
||||
log.Printf("Cleanup: failed to scan %q for expired files: %v", uploadDir, err)
|
||||
}
|
||||
|
||||
removedFiles := removeAll(oldFiles, "expired file", dryRun)
|
||||
|
||||
emptyDirs, err := collectEmptyDirs(uploadDir)
|
||||
if err != nil {
|
||||
log.Printf("Cleanup: failed to scan %q for empty folders: %v", uploadDir, err)
|
||||
}
|
||||
|
||||
removedDirs := removeAll(emptyDirs, "empty folder", dryRun)
|
||||
|
||||
log.Printf("Cleanup finished: %d expired file(s) and %d empty folder(s) found, %d and %d removed (dryRun=%v)",
|
||||
len(oldFiles), len(emptyDirs), removedFiles, removedDirs, dryRun)
|
||||
}
|
||||
|
||||
// collectExpiredFiles returns files under root whose mtime is at least
|
||||
// duration old.
|
||||
func collectExpiredFiles(root string, duration time.Duration) ([]string, error) {
|
||||
var oldFiles []string
|
||||
|
||||
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
log.Printf("Cleanup: skipping %q: %v", path, err)
|
||||
return nil
|
||||
}
|
||||
if d.IsDir() {
|
||||
@@ -490,64 +751,76 @@ func cleanupProcedure(duration time.Duration, dryRun bool) {
|
||||
|
||||
info, err := d.Info()
|
||||
if err != nil {
|
||||
log.Printf("Error cleaning file '%s' : %v", d.Name(), err)
|
||||
log.Printf("Cleanup: failed to stat %q: %v", path, err)
|
||||
return nil
|
||||
}
|
||||
age := time.Since(info.ModTime())
|
||||
if age >= duration {
|
||||
if time.Since(info.ModTime()) >= duration {
|
||||
oldFiles = append(oldFiles, path)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
})
|
||||
|
||||
findEmptyDirs := func(path string, d fs.DirEntry, err error) error {
|
||||
return oldFiles, err
|
||||
}
|
||||
|
||||
// collectEmptyDirs returns empty directories under root, excluding root itself
|
||||
// and any directory created within newUploadGrace, which may belong to an
|
||||
// upload that has not written its file yet.
|
||||
func collectEmptyDirs(root string) ([]string, error) {
|
||||
var emptyDirs []string
|
||||
|
||||
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
|
||||
// d is nil when the root itself cannot be read, so this check must
|
||||
// come first.
|
||||
if err != nil {
|
||||
log.Printf("Cleanup: skipping %q: %v", path, err)
|
||||
return nil
|
||||
}
|
||||
if !d.IsDir() {
|
||||
return nil
|
||||
}
|
||||
if filepath.Clean(path) == filepath.Clean(uploadDir) {
|
||||
if filepath.Clean(path) == filepath.Clean(root) {
|
||||
return nil
|
||||
}
|
||||
|
||||
info, err := d.Info()
|
||||
if err != nil {
|
||||
log.Printf("Cleanup: failed to stat %q: %v", path, err)
|
||||
return nil
|
||||
}
|
||||
if time.Since(info.ModTime()) < newUploadGrace {
|
||||
return nil
|
||||
}
|
||||
|
||||
empty, err := isDirEmpty(path)
|
||||
if err != nil {
|
||||
log.Printf("Cleanup: failed to read %q: %v", path, err)
|
||||
return nil
|
||||
}
|
||||
if empty {
|
||||
emptyDirs = append(emptyDirs, path)
|
||||
} else if err != nil {
|
||||
log.Printf("Error cleaning directory '%s' : %v", path, err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
})
|
||||
|
||||
for {
|
||||
oldFiles = oldFiles[:0]
|
||||
emptyDirs = emptyDirs[:0]
|
||||
return emptyDirs, err
|
||||
}
|
||||
|
||||
filepath.WalkDir(uploadDir, deleteIfTooOld)
|
||||
for _, file := range oldFiles {
|
||||
if !dryRun {
|
||||
err := os.Remove(file)
|
||||
if err != nil {
|
||||
log.Printf("Failed to remove expired file '%s' : %v", file, err)
|
||||
} else {
|
||||
log.Printf("Removed expired file '%s'", file)
|
||||
// removeAll deletes the given paths unless dryRun is set, and returns how many
|
||||
// were actually removed.
|
||||
func removeAll(paths []string, kind string, dryRun bool) int {
|
||||
removed := 0
|
||||
for _, path := range paths {
|
||||
if dryRun {
|
||||
log.Printf("Dry run: Would remove %s %q", kind, path)
|
||||
continue
|
||||
}
|
||||
} else {
|
||||
log.Printf("Dry run: Would remove expired file '%s'", file)
|
||||
if err := os.Remove(path); err != nil {
|
||||
log.Printf("Failed to remove %s %q: %v", kind, path, err)
|
||||
continue
|
||||
}
|
||||
log.Printf("Removed %s %q", kind, path)
|
||||
removed++
|
||||
}
|
||||
|
||||
filepath.WalkDir(uploadDir, findEmptyDirs)
|
||||
for _, file := range emptyDirs {
|
||||
if !dryRun {
|
||||
err := os.Remove(file)
|
||||
if err != nil {
|
||||
log.Printf("Failed to remove empty folder '%s' : %v", file, err)
|
||||
} else {
|
||||
log.Printf("Removed empty folder '%s'", file)
|
||||
}
|
||||
} else {
|
||||
log.Printf("Dry run: Would remove empty folder '%s'", file)
|
||||
}
|
||||
}
|
||||
time.Sleep(6 * time.Hour)
|
||||
}
|
||||
return removed
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user