Compare commits
11
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8ce8db11f9 | ||
|
|
ef29ac04dc | ||
|
|
34b90c62ca | ||
|
|
00920d42ef | ||
|
|
2f91a056d5 | ||
|
|
5817269855 | ||
|
|
b774e23ac0 | ||
|
|
b81d74220e | ||
|
|
5fcd2b4473 | ||
|
|
c0b4fa2617 | ||
|
|
a428727034 |
+97
@@ -0,0 +1,97 @@
|
||||
# Cline rules for the "beam" project
|
||||
|
||||
## Role
|
||||
|
||||
Act as an expert Go backend/web developer. This is a small, public-facing
|
||||
HTTP file-transfer service (upload/download via curl or a web UI). Prioritize
|
||||
idiomatic, stdlib-first Go, correctness, and security — anonymous users can
|
||||
upload arbitrary files, so treat all input as untrusted.
|
||||
|
||||
## Project architecture facts
|
||||
|
||||
- Single-file `main.go` using only the Go standard library (`net/http`,
|
||||
`html/template`, etc.) — no web framework, router, or third-party
|
||||
dependencies. Keep `go.mod` dependency-free unless the user explicitly
|
||||
approves adding one.
|
||||
- `index.html.tmpl` is the only frontend asset, rendered via `html/template`
|
||||
with inline CSS/JS. Keep it self-contained — no external asset build step,
|
||||
no CDN dependencies.
|
||||
- Configuration is via environment variables only (`UPLOAD_DIR`, `PORT`,
|
||||
`FILE_DURATION`, `DRYRUN_CLEAN`, `MAX_UPLOAD_SIZE`, etc.). Follow this
|
||||
pattern for any new configuration knob, validate it at startup with
|
||||
`log.Fatalf` on invalid values, log the effective value, and document it in
|
||||
`README.md`.
|
||||
- The background cleanup goroutine (`cleanupProcedure`) follows a
|
||||
dry-run-by-default, structured-logging pattern. Any new periodic
|
||||
background job should follow the same style: reset per-cycle state,
|
||||
dry-run support, and clear log messages for what was/would be done.
|
||||
|
||||
## Go version
|
||||
|
||||
- The toolchain version is pinned in `go.mod` (currently `go 1.27`). Match
|
||||
`go.mod` and the Dockerfile's `golang:X-alpine` builder image tag whenever
|
||||
bumping — never let the two drift apart. Don't use stdlib APIs newer than
|
||||
the pinned version.
|
||||
|
||||
## Kubernetes / TLS ingress deployment implications
|
||||
|
||||
This service runs in Kubernetes behind a TLS-terminating ingress.
|
||||
|
||||
- Never rely on `r.TLS != nil` to detect HTTPS. Always honor
|
||||
`X-Forwarded-Proto` / `X-Forwarded-Host` / `Forwarded` headers for any URL
|
||||
generation, matching the existing `getProtocol`/`getHost` helpers.
|
||||
- `/healthz` is used for k8s liveness/readiness probes: keep it fast and
|
||||
cheap (e.g. `os.Stat` on the upload dir is fine). Do not add slow checks,
|
||||
external calls, or heavy I/O to it.
|
||||
- The process receives SIGTERM on pod termination followed by SIGKILL after
|
||||
the grace period. Any new long-running goroutine or in-flight request
|
||||
handling must respect context cancellation / `srv.Shutdown` and not block
|
||||
graceful shutdown.
|
||||
- Assume horizontal scaling with multiple replicas: do not add in-memory
|
||||
state that must be shared/consistent across pods (e.g. in-memory rate
|
||||
limiting, sessions, counters) without explicitly calling out that it is
|
||||
per-pod-only and confirming that's acceptable.
|
||||
- `UPLOAD_DIR` may be backed by a PersistentVolume. Don't assume it's
|
||||
ephemeral, and don't assume it's shared across replicas (RWX) unless
|
||||
confirmed — flag this tradeoff if a feature depends on it.
|
||||
- The container runs as a non-root user (see Dockerfile `USER beam`). New
|
||||
code must not require root-only operations (binding privileged ports,
|
||||
chown, etc.).
|
||||
|
||||
## Coding conventions
|
||||
|
||||
- Always run `go build ./...`, `go vet ./...`, and `gofmt -l .` before
|
||||
considering a change complete; fix any findings.
|
||||
- Prefer the standard library over new dependencies (e.g. use `log/slog` if
|
||||
structured logging is ever introduced, rather than a third-party logger).
|
||||
- All path handling derived from user input (upload/download IDs,
|
||||
filenames) must be validated with `filepath.IsLocal` and/or
|
||||
`filepath.Clean`/`filepath.Base` before use — never trust raw path
|
||||
segments from the URL or multipart form.
|
||||
- All upload code paths must enforce `maxUploadSize` via
|
||||
`http.MaxBytesReader`, and clean up any partially written file/directory
|
||||
on failure.
|
||||
- New HTTP handlers must call `logRequest` with an accurate status code for
|
||||
every response path.
|
||||
- Keep the JSON error shape for `/upload` consistent with
|
||||
`writeMultipartError` (`{"error": "..."}`).
|
||||
|
||||
## Git workflow
|
||||
|
||||
- One logical change per commit.
|
||||
- Commit subject is a single line, no body.
|
||||
- Keep `.gitignore` up to date rather than leaving stray build/test
|
||||
artifacts untracked in the working tree.
|
||||
|
||||
## Testing
|
||||
|
||||
- There are currently no `*_test.go` files. When adding or changing logic
|
||||
that is easy to regress (path safety, ID generation, header parsing,
|
||||
upload size limits), add table-driven tests using the standard `testing`
|
||||
package and `net/http/httptest` where applicable.
|
||||
|
||||
## Documentation
|
||||
|
||||
- Keep `README.md` and the in-page instructions/labels in
|
||||
`index.html.tmpl` (e.g. max upload size, usage examples) in sync with
|
||||
actual runtime behavior.
|
||||
+1
-1
@@ -1,4 +1,4 @@
|
||||
FROM golang:1.26-alpine AS builder
|
||||
FROM golang:1.27-alpine AS builder
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
|
||||
+56
-6
@@ -57,7 +57,7 @@
|
||||
}
|
||||
button:hover { background: #555; }
|
||||
.progress-container {
|
||||
width: 100%%;
|
||||
width: 100%;
|
||||
background: #ddd;
|
||||
border-radius: 4px;
|
||||
margin: 10px 0;
|
||||
@@ -65,7 +65,7 @@
|
||||
display: none;
|
||||
}
|
||||
.progress-bar {
|
||||
height: 100%%;
|
||||
height: 100%;
|
||||
background: #333;
|
||||
border-radius: 4px;
|
||||
width: 0;
|
||||
@@ -93,6 +93,7 @@
|
||||
<h2>Upload with curl</h2>
|
||||
<code>curl --upload-file myfile.txt {{ .Proto }}://{{ .Host }}/myfile.txt</code>
|
||||
<p style="margin-top: 10px;">The server will respond with a download URL.</p>
|
||||
<p style="margin-top: 10px;">Maximum upload size: <strong id="maxUploadSizeLabel" data-max-bytes="{{ .MaxUploadSize }}">{{ .MaxUploadSize }} bytes</strong></p>
|
||||
</div>
|
||||
|
||||
<div class="section">
|
||||
@@ -105,6 +106,7 @@
|
||||
<h2>Web Upload</h2>
|
||||
<div class="upload-area" id="uploadArea">
|
||||
<p>📁 Click to select files or drag and drop here</p>
|
||||
<p style="margin-top: 10px; font-size: 0.85em; color: #666;">Maximum file size: <span id="maxUploadSizeText"></span></p>
|
||||
<input type="file" id="fileInput" multiple>
|
||||
<button onclick="document.getElementById('fileInput').click()">Select Files</button>
|
||||
</div>
|
||||
@@ -120,6 +122,22 @@
|
||||
const progressContainer = document.getElementById('progressContainer');
|
||||
const progressBar = document.getElementById('progressBar');
|
||||
const results = document.getElementById('results');
|
||||
const maxUploadSize = parseInt(document.getElementById('maxUploadSizeLabel').dataset.maxBytes, 10) || 0;
|
||||
|
||||
function formatBytes(bytes) {
|
||||
if (!bytes) return 'unknown';
|
||||
const units = ['B', 'KB', 'MB', 'GB', 'TB'];
|
||||
let value = bytes;
|
||||
let unitIndex = 0;
|
||||
while (value >= 1024 && unitIndex < units.length - 1) {
|
||||
value /= 1024;
|
||||
unitIndex++;
|
||||
}
|
||||
return value.toFixed(unitIndex === 0 ? 0 : 1) + ' ' + units[unitIndex];
|
||||
}
|
||||
|
||||
document.getElementById('maxUploadSizeLabel').textContent = formatBytes(maxUploadSize);
|
||||
document.getElementById('maxUploadSizeText').textContent = formatBytes(maxUploadSize);
|
||||
|
||||
uploadArea.addEventListener('click', () => fileInput.click());
|
||||
|
||||
@@ -146,13 +164,34 @@
|
||||
function uploadFiles(files) {
|
||||
if (files.length === 0) return;
|
||||
|
||||
const oversizedFiles = [];
|
||||
let totalSize = 0;
|
||||
for (let file of files) {
|
||||
totalSize += file.size;
|
||||
if (maxUploadSize && file.size > maxUploadSize) {
|
||||
oversizedFiles.push(file.name);
|
||||
}
|
||||
}
|
||||
|
||||
if (oversizedFiles.length > 0) {
|
||||
results.innerHTML = '<div class="result-item">Upload failed: the following file(s) exceed the maximum size of ' +
|
||||
formatBytes(maxUploadSize) + ': ' + oversizedFiles.join(', ') + '</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
if (maxUploadSize && totalSize > maxUploadSize) {
|
||||
results.innerHTML = '<div class="result-item">Upload failed: total upload size (' + formatBytes(totalSize) +
|
||||
') exceeds the maximum of ' + formatBytes(maxUploadSize) + '</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
const formData = new FormData();
|
||||
for (let file of files) {
|
||||
formData.append('files', file);
|
||||
}
|
||||
|
||||
progressContainer.style.display = 'block';
|
||||
progressBar.style.width = '0%%';
|
||||
progressBar.style.width = '0%';
|
||||
results.innerHTML = '';
|
||||
|
||||
const xhr = new XMLHttpRequest();
|
||||
@@ -160,7 +199,7 @@
|
||||
xhr.upload.addEventListener('progress', (e) => {
|
||||
if (e.lengthComputable) {
|
||||
const percent = (e.loaded / e.total) * 100;
|
||||
progressBar.style.width = percent + '%%';
|
||||
progressBar.style.width = percent + '%';
|
||||
}
|
||||
});
|
||||
|
||||
@@ -169,9 +208,20 @@
|
||||
const response = JSON.parse(xhr.responseText);
|
||||
displayResults(response.urls);
|
||||
} else {
|
||||
results.innerHTML = '<div class="result-item">Upload failed: ' + xhr.statusText + '</div>';
|
||||
let message = xhr.statusText;
|
||||
try {
|
||||
const response = JSON.parse(xhr.responseText);
|
||||
if (response.error) {
|
||||
message = response.error;
|
||||
}
|
||||
progressBar.style.width = '100%%';
|
||||
} catch (e) {
|
||||
if (xhr.responseText) {
|
||||
message = xhr.responseText;
|
||||
}
|
||||
}
|
||||
results.innerHTML = '<div class="result-item">Upload failed: ' + message + '</div>';
|
||||
}
|
||||
progressBar.style.width = '100%';
|
||||
});
|
||||
|
||||
xhr.addEventListener('error', () => {
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
@@ -12,8 +13,11 @@ import (
|
||||
"mime"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
@@ -22,7 +26,16 @@ var uploadDir string
|
||||
const idLength = 8
|
||||
const idCharset = "abcdefghijklmnopqrstuvwxyz0123456789"
|
||||
|
||||
// defaultMaxUploadSize is used when MAX_UPLOAD_SIZE is not set.
|
||||
const defaultMaxUploadSize int64 = 5 << 30 // 5GB
|
||||
|
||||
// maxMultipartMemory caps how much of a multipart upload is buffered in
|
||||
// memory before spilling to temporary files on disk, regardless of
|
||||
// maxUploadSize.
|
||||
const maxMultipartMemory int64 = 32 << 20 // 32MB
|
||||
|
||||
var indexHTML *template.Template
|
||||
var maxUploadSize int64 = defaultMaxUploadSize
|
||||
|
||||
func main() {
|
||||
uploadDir = os.Getenv("UPLOAD_DIR")
|
||||
@@ -41,6 +54,14 @@ func main() {
|
||||
fileExpires = expires
|
||||
}
|
||||
|
||||
if maxUploadSizeEnv := os.Getenv("MAX_UPLOAD_SIZE"); maxUploadSizeEnv != "" {
|
||||
size, err := strconv.ParseInt(maxUploadSizeEnv, 10, 64)
|
||||
if err != nil || size <= 0 {
|
||||
log.Fatalf("MAX_UPLOAD_SIZE %s is invalid: %v", maxUploadSizeEnv, err)
|
||||
}
|
||||
maxUploadSize = size
|
||||
}
|
||||
|
||||
port := os.Getenv("PORT")
|
||||
if port == "" {
|
||||
port = "8080"
|
||||
@@ -54,14 +75,45 @@ func main() {
|
||||
|
||||
log.Printf("Starting server on port %s, upload directory: %s", port, uploadDir)
|
||||
log.Printf("Files expires after: %v (DRYRUN_CLEAN=%v)", fileExpires, dryRun)
|
||||
log.Printf("Max upload size: %d bytes", maxUploadSize)
|
||||
if fileExpires != 0 {
|
||||
go cleanupProcedure(fileExpires, dryRun)
|
||||
} else {
|
||||
log.Print("Cleaning old files is disabled")
|
||||
}
|
||||
|
||||
if err := http.ListenAndServe(":"+port, nil); err != nil {
|
||||
// Note: ReadTimeout/WriteTimeout are intentionally not set since they
|
||||
// would cap the total duration of large file uploads/downloads.
|
||||
// ReadHeaderTimeout and IdleTimeout still protect against slow-loris
|
||||
// style attacks on connection/header handling.
|
||||
srv := &http.Server{
|
||||
Addr: ":" + port,
|
||||
ReadHeaderTimeout: 10 * time.Second,
|
||||
IdleTimeout: 120 * time.Second,
|
||||
}
|
||||
|
||||
serverErr := make(chan error, 1)
|
||||
go func() {
|
||||
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
|
||||
serverErr <- err
|
||||
}
|
||||
close(serverErr)
|
||||
}()
|
||||
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
|
||||
select {
|
||||
case err := <-serverErr:
|
||||
log.Fatalf("Server failed: %v", err)
|
||||
case <-ctx.Done():
|
||||
log.Print("Shutdown signal received, shutting down gracefully...")
|
||||
shutdownCtx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
|
||||
defer cancel()
|
||||
if err := srv.Shutdown(shutdownCtx); err != nil {
|
||||
log.Fatalf("Graceful shutdown failed: %v", err)
|
||||
}
|
||||
log.Print("Server stopped")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -74,6 +126,7 @@ func handleRequest(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
if r.Method == "GET" && r.URL.Path == "/healthz" {
|
||||
handleHealthz(w, r)
|
||||
return
|
||||
}
|
||||
|
||||
@@ -100,6 +153,15 @@ func handleRequest(w http.ResponseWriter, r *http.Request) {
|
||||
logRequest(r, 405, time.Since(start))
|
||||
}
|
||||
|
||||
func handleHealthz(w http.ResponseWriter, r *http.Request) {
|
||||
if _, err := os.Stat(uploadDir); err != nil {
|
||||
log.Printf("Healthz check failed: %v", err)
|
||||
http.Error(w, "Upload directory is not accessible", http.StatusServiceUnavailable)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}
|
||||
|
||||
func handleUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
|
||||
filename := strings.TrimPrefix(r.URL.Path, "/")
|
||||
if filename == "" {
|
||||
@@ -126,8 +188,16 @@ func handleUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
written, err := io.Copy(file, r.Body)
|
||||
limitedBody := http.MaxBytesReader(w, r.Body, maxUploadSize)
|
||||
written, err := io.Copy(file, limitedBody)
|
||||
if err != nil {
|
||||
file.Close()
|
||||
os.Remove(targetPath)
|
||||
if isMaxBytesError(err) {
|
||||
http.Error(w, fmt.Sprintf("File exceeds maximum upload size of %d bytes", maxUploadSize), http.StatusRequestEntityTooLarge)
|
||||
logRequest(r, 413, time.Since(start))
|
||||
return
|
||||
}
|
||||
http.Error(w, "Failed to write file", http.StatusInternalServerError)
|
||||
logRequest(r, 500, time.Since(start))
|
||||
return
|
||||
@@ -144,16 +214,34 @@ func handleUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
|
||||
logRequest(r, 200, time.Since(start))
|
||||
}
|
||||
|
||||
func writeMultipartError(w http.ResponseWriter, status int, message string) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
json.NewEncoder(w).Encode(map[string]string{"error": message})
|
||||
}
|
||||
|
||||
func handleMultipartUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
|
||||
if err := r.ParseMultipartForm(32 << 20); err != nil {
|
||||
http.Error(w, "Failed to parse multipart form", http.StatusBadRequest)
|
||||
r.Body = http.MaxBytesReader(w, r.Body, maxUploadSize)
|
||||
|
||||
memoryLimit := maxUploadSize
|
||||
if memoryLimit > maxMultipartMemory {
|
||||
memoryLimit = maxMultipartMemory
|
||||
}
|
||||
|
||||
if err := r.ParseMultipartForm(memoryLimit); err != nil {
|
||||
if isMaxBytesError(err) {
|
||||
writeMultipartError(w, http.StatusRequestEntityTooLarge, fmt.Sprintf("Upload exceeds maximum size of %d bytes", maxUploadSize))
|
||||
logRequest(r, 413, time.Since(start))
|
||||
return
|
||||
}
|
||||
writeMultipartError(w, http.StatusBadRequest, "Failed to parse multipart form")
|
||||
logRequest(r, 400, time.Since(start))
|
||||
return
|
||||
}
|
||||
|
||||
files := r.MultipartForm.File["files"]
|
||||
if len(files) == 0 {
|
||||
http.Error(w, "No files provided", http.StatusBadRequest)
|
||||
writeMultipartError(w, http.StatusBadRequest, "No files provided")
|
||||
logRequest(r, 400, time.Since(start))
|
||||
return
|
||||
}
|
||||
@@ -188,6 +276,7 @@ func handleMultipartUpload(w http.ResponseWriter, r *http.Request, start time.Ti
|
||||
src.Close()
|
||||
|
||||
if err != nil {
|
||||
os.Remove(targetPath)
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -212,6 +301,12 @@ func handleDownload(w http.ResponseWriter, r *http.Request, start time.Time) {
|
||||
id := parts[0]
|
||||
filename := parts[1]
|
||||
|
||||
if !filepath.IsLocal(id) || !filepath.IsLocal(filename) {
|
||||
http.NotFound(w, r)
|
||||
logRequest(r, 404, time.Since(start))
|
||||
return
|
||||
}
|
||||
|
||||
filePath := filepath.Join(uploadDir, id, filename)
|
||||
file, err := os.Open(filePath)
|
||||
if err != nil {
|
||||
@@ -219,12 +314,6 @@ func handleDownload(w http.ResponseWriter, r *http.Request, start time.Time) {
|
||||
logRequest(r, 404, time.Since(start))
|
||||
return
|
||||
}
|
||||
if !strings.HasPrefix(filePath, uploadDir) {
|
||||
http.NotFound(w, r)
|
||||
logRequest(r, 404, time.Since(start))
|
||||
return
|
||||
}
|
||||
|
||||
defer file.Close()
|
||||
|
||||
stat, err := file.Stat()
|
||||
@@ -249,11 +338,10 @@ func generateUniqueID() (string, error) {
|
||||
id := generateID()
|
||||
targetDir := filepath.Join(uploadDir, id)
|
||||
|
||||
if _, err := os.Stat(targetDir); os.IsNotExist(err) {
|
||||
if err := os.MkdirAll(targetDir, 0755); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if err := os.Mkdir(targetDir, 0755); err == nil {
|
||||
return id, nil
|
||||
} else if !os.IsExist(err) {
|
||||
return "", err
|
||||
}
|
||||
}
|
||||
|
||||
@@ -324,6 +412,13 @@ func getProtocol(r *http.Request) string {
|
||||
return "http"
|
||||
}
|
||||
|
||||
// isMaxBytesError reports whether err was caused by exceeding the limit
|
||||
// set on an http.MaxBytesReader.
|
||||
func isMaxBytesError(err error) bool {
|
||||
var maxBytesErr *http.MaxBytesError
|
||||
return errors.As(err, &maxBytesErr)
|
||||
}
|
||||
|
||||
func detectContentType(filename string) string {
|
||||
ext := filepath.Ext(filename)
|
||||
mimeType := mime.TypeByExtension(ext)
|
||||
@@ -340,7 +435,11 @@ func logRequest(r *http.Request, status int, duration time.Duration) {
|
||||
func serveFrontpage(w http.ResponseWriter, r *http.Request) {
|
||||
proto := getProtocol(r)
|
||||
host := getHost(r)
|
||||
pageData := map[string]string{"Proto": proto, "Host": host}
|
||||
pageData := map[string]any{
|
||||
"Proto": proto,
|
||||
"Host": host,
|
||||
"MaxUploadSize": maxUploadSize,
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
indexHTML.Execute(w, pageData)
|
||||
@@ -395,6 +494,9 @@ func cleanupProcedure(duration time.Duration, dryRun bool) {
|
||||
if !d.IsDir() {
|
||||
return nil
|
||||
}
|
||||
if filepath.Clean(path) == filepath.Clean(uploadDir) {
|
||||
return nil
|
||||
}
|
||||
empty, err := isDirEmpty(path)
|
||||
if empty {
|
||||
emptyDirs = append(emptyDirs, path)
|
||||
@@ -406,6 +508,8 @@ func cleanupProcedure(duration time.Duration, dryRun bool) {
|
||||
}
|
||||
|
||||
for {
|
||||
oldFiles = oldFiles[:0]
|
||||
emptyDirs = emptyDirs[:0]
|
||||
|
||||
filepath.WalkDir(uploadDir, deleteIfTooOld)
|
||||
for _, file := range oldFiles {
|
||||
|
||||
Reference in New Issue
Block a user