Compare commits

..
11 Commits
5 changed files with 275 additions and 24 deletions
+97
View File
@@ -0,0 +1,97 @@
# Cline rules for the "beam" project
## Role
Act as an expert Go backend/web developer. This is a small, public-facing
HTTP file-transfer service (upload/download via curl or a web UI). Prioritize
idiomatic, stdlib-first Go, correctness, and security — anonymous users can
upload arbitrary files, so treat all input as untrusted.
## Project architecture facts
- Single-file `main.go` using only the Go standard library (`net/http`,
`html/template`, etc.) — no web framework, router, or third-party
dependencies. Keep `go.mod` dependency-free unless the user explicitly
approves adding one.
- `index.html.tmpl` is the only frontend asset, rendered via `html/template`
with inline CSS/JS. Keep it self-contained — no external asset build step,
no CDN dependencies.
- Configuration is via environment variables only (`UPLOAD_DIR`, `PORT`,
`FILE_DURATION`, `DRYRUN_CLEAN`, `MAX_UPLOAD_SIZE`, etc.). Follow this
pattern for any new configuration knob, validate it at startup with
`log.Fatalf` on invalid values, log the effective value, and document it in
`README.md`.
- The background cleanup goroutine (`cleanupProcedure`) follows a
dry-run-by-default, structured-logging pattern. Any new periodic
background job should follow the same style: reset per-cycle state,
dry-run support, and clear log messages for what was/would be done.
## Go version
- The toolchain version is pinned in `go.mod` (currently `go 1.27`). Match
`go.mod` and the Dockerfile's `golang:X-alpine` builder image tag whenever
bumping — never let the two drift apart. Don't use stdlib APIs newer than
the pinned version.
## Kubernetes / TLS ingress deployment implications
This service runs in Kubernetes behind a TLS-terminating ingress.
- Never rely on `r.TLS != nil` to detect HTTPS. Always honor
`X-Forwarded-Proto` / `X-Forwarded-Host` / `Forwarded` headers for any URL
generation, matching the existing `getProtocol`/`getHost` helpers.
- `/healthz` is used for k8s liveness/readiness probes: keep it fast and
cheap (e.g. `os.Stat` on the upload dir is fine). Do not add slow checks,
external calls, or heavy I/O to it.
- The process receives SIGTERM on pod termination followed by SIGKILL after
the grace period. Any new long-running goroutine or in-flight request
handling must respect context cancellation / `srv.Shutdown` and not block
graceful shutdown.
- Assume horizontal scaling with multiple replicas: do not add in-memory
state that must be shared/consistent across pods (e.g. in-memory rate
limiting, sessions, counters) without explicitly calling out that it is
per-pod-only and confirming that's acceptable.
- `UPLOAD_DIR` may be backed by a PersistentVolume. Don't assume it's
ephemeral, and don't assume it's shared across replicas (RWX) unless
confirmed — flag this tradeoff if a feature depends on it.
- The container runs as a non-root user (see Dockerfile `USER beam`). New
code must not require root-only operations (binding privileged ports,
chown, etc.).
## Coding conventions
- Always run `go build ./...`, `go vet ./...`, and `gofmt -l .` before
considering a change complete; fix any findings.
- Prefer the standard library over new dependencies (e.g. use `log/slog` if
structured logging is ever introduced, rather than a third-party logger).
- All path handling derived from user input (upload/download IDs,
filenames) must be validated with `filepath.IsLocal` and/or
`filepath.Clean`/`filepath.Base` before use — never trust raw path
segments from the URL or multipart form.
- All upload code paths must enforce `maxUploadSize` via
`http.MaxBytesReader`, and clean up any partially written file/directory
on failure.
- New HTTP handlers must call `logRequest` with an accurate status code for
every response path.
- Keep the JSON error shape for `/upload` consistent with
`writeMultipartError` (`{"error": "..."}`).
## Git workflow
- One logical change per commit.
- Commit subject is a single line, no body.
- Keep `.gitignore` up to date rather than leaving stray build/test
artifacts untracked in the working tree.
## Testing
- There are currently no `*_test.go` files. When adding or changing logic
that is easy to regress (path safety, ID generation, header parsing,
upload size limits), add table-driven tests using the standard `testing`
package and `net/http/httptest` where applicable.
## Documentation
- Keep `README.md` and the in-page instructions/labels in
`index.html.tmpl` (e.g. max upload size, usage examples) in sync with
actual runtime behavior.
+1 -1
View File
@@ -1,4 +1,4 @@
FROM golang:1.26-alpine AS builder
FROM golang:1.27-alpine AS builder
WORKDIR /build
+1 -1
View File
@@ -1,3 +1,3 @@
module beam
go 1.24
go 1.27
+56 -6
View File
@@ -57,7 +57,7 @@
}
button:hover { background: #555; }
.progress-container {
width: 100%%;
width: 100%;
background: #ddd;
border-radius: 4px;
margin: 10px 0;
@@ -65,7 +65,7 @@
display: none;
}
.progress-bar {
height: 100%%;
height: 100%;
background: #333;
border-radius: 4px;
width: 0;
@@ -93,6 +93,7 @@
<h2>Upload with curl</h2>
<code>curl --upload-file myfile.txt {{ .Proto }}://{{ .Host }}/myfile.txt</code>
<p style="margin-top: 10px;">The server will respond with a download URL.</p>
<p style="margin-top: 10px;">Maximum upload size: <strong id="maxUploadSizeLabel" data-max-bytes="{{ .MaxUploadSize }}">{{ .MaxUploadSize }} bytes</strong></p>
</div>
<div class="section">
@@ -105,6 +106,7 @@
<h2>Web Upload</h2>
<div class="upload-area" id="uploadArea">
<p>📁 Click to select files or drag and drop here</p>
<p style="margin-top: 10px; font-size: 0.85em; color: #666;">Maximum file size: <span id="maxUploadSizeText"></span></p>
<input type="file" id="fileInput" multiple>
<button onclick="document.getElementById('fileInput').click()">Select Files</button>
</div>
@@ -120,6 +122,22 @@
const progressContainer = document.getElementById('progressContainer');
const progressBar = document.getElementById('progressBar');
const results = document.getElementById('results');
const maxUploadSize = parseInt(document.getElementById('maxUploadSizeLabel').dataset.maxBytes, 10) || 0;
function formatBytes(bytes) {
if (!bytes) return 'unknown';
const units = ['B', 'KB', 'MB', 'GB', 'TB'];
let value = bytes;
let unitIndex = 0;
while (value >= 1024 && unitIndex < units.length - 1) {
value /= 1024;
unitIndex++;
}
return value.toFixed(unitIndex === 0 ? 0 : 1) + ' ' + units[unitIndex];
}
document.getElementById('maxUploadSizeLabel').textContent = formatBytes(maxUploadSize);
document.getElementById('maxUploadSizeText').textContent = formatBytes(maxUploadSize);
uploadArea.addEventListener('click', () => fileInput.click());
@@ -146,13 +164,34 @@
function uploadFiles(files) {
if (files.length === 0) return;
const oversizedFiles = [];
let totalSize = 0;
for (let file of files) {
totalSize += file.size;
if (maxUploadSize && file.size > maxUploadSize) {
oversizedFiles.push(file.name);
}
}
if (oversizedFiles.length > 0) {
results.innerHTML = '<div class="result-item">Upload failed: the following file(s) exceed the maximum size of ' +
formatBytes(maxUploadSize) + ': ' + oversizedFiles.join(', ') + '</div>';
return;
}
if (maxUploadSize && totalSize > maxUploadSize) {
results.innerHTML = '<div class="result-item">Upload failed: total upload size (' + formatBytes(totalSize) +
') exceeds the maximum of ' + formatBytes(maxUploadSize) + '</div>';
return;
}
const formData = new FormData();
for (let file of files) {
formData.append('files', file);
}
progressContainer.style.display = 'block';
progressBar.style.width = '0%%';
progressBar.style.width = '0%';
results.innerHTML = '';
const xhr = new XMLHttpRequest();
@@ -160,7 +199,7 @@
xhr.upload.addEventListener('progress', (e) => {
if (e.lengthComputable) {
const percent = (e.loaded / e.total) * 100;
progressBar.style.width = percent + '%%';
progressBar.style.width = percent + '%';
}
});
@@ -169,9 +208,20 @@
const response = JSON.parse(xhr.responseText);
displayResults(response.urls);
} else {
results.innerHTML = '<div class="result-item">Upload failed: ' + xhr.statusText + '</div>';
let message = xhr.statusText;
try {
const response = JSON.parse(xhr.responseText);
if (response.error) {
message = response.error;
}
progressBar.style.width = '100%%';
} catch (e) {
if (xhr.responseText) {
message = xhr.responseText;
}
}
results.innerHTML = '<div class="result-item">Upload failed: ' + message + '</div>';
}
progressBar.style.width = '100%';
});
xhr.addEventListener('error', () => {
+120 -16
View File
@@ -1,6 +1,7 @@
package main
import (
"context"
"crypto/rand"
"encoding/json"
"errors"
@@ -12,8 +13,11 @@ import (
"mime"
"net/http"
"os"
"os/signal"
"path/filepath"
"strconv"
"strings"
"syscall"
"time"
)
@@ -22,7 +26,16 @@ var uploadDir string
const idLength = 8
const idCharset = "abcdefghijklmnopqrstuvwxyz0123456789"
// defaultMaxUploadSize is used when MAX_UPLOAD_SIZE is not set.
const defaultMaxUploadSize int64 = 5 << 30 // 5GB
// maxMultipartMemory caps how much of a multipart upload is buffered in
// memory before spilling to temporary files on disk, regardless of
// maxUploadSize.
const maxMultipartMemory int64 = 32 << 20 // 32MB
var indexHTML *template.Template
var maxUploadSize int64 = defaultMaxUploadSize
func main() {
uploadDir = os.Getenv("UPLOAD_DIR")
@@ -41,6 +54,14 @@ func main() {
fileExpires = expires
}
if maxUploadSizeEnv := os.Getenv("MAX_UPLOAD_SIZE"); maxUploadSizeEnv != "" {
size, err := strconv.ParseInt(maxUploadSizeEnv, 10, 64)
if err != nil || size <= 0 {
log.Fatalf("MAX_UPLOAD_SIZE %s is invalid: %v", maxUploadSizeEnv, err)
}
maxUploadSize = size
}
port := os.Getenv("PORT")
if port == "" {
port = "8080"
@@ -54,14 +75,45 @@ func main() {
log.Printf("Starting server on port %s, upload directory: %s", port, uploadDir)
log.Printf("Files expires after: %v (DRYRUN_CLEAN=%v)", fileExpires, dryRun)
log.Printf("Max upload size: %d bytes", maxUploadSize)
if fileExpires != 0 {
go cleanupProcedure(fileExpires, dryRun)
} else {
log.Print("Cleaning old files is disabled")
}
if err := http.ListenAndServe(":"+port, nil); err != nil {
// Note: ReadTimeout/WriteTimeout are intentionally not set since they
// would cap the total duration of large file uploads/downloads.
// ReadHeaderTimeout and IdleTimeout still protect against slow-loris
// style attacks on connection/header handling.
srv := &http.Server{
Addr: ":" + port,
ReadHeaderTimeout: 10 * time.Second,
IdleTimeout: 120 * time.Second,
}
serverErr := make(chan error, 1)
go func() {
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
serverErr <- err
}
close(serverErr)
}()
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
select {
case err := <-serverErr:
log.Fatalf("Server failed: %v", err)
case <-ctx.Done():
log.Print("Shutdown signal received, shutting down gracefully...")
shutdownCtx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
if err := srv.Shutdown(shutdownCtx); err != nil {
log.Fatalf("Graceful shutdown failed: %v", err)
}
log.Print("Server stopped")
}
}
@@ -74,6 +126,7 @@ func handleRequest(w http.ResponseWriter, r *http.Request) {
return
}
if r.Method == "GET" && r.URL.Path == "/healthz" {
handleHealthz(w, r)
return
}
@@ -100,6 +153,15 @@ func handleRequest(w http.ResponseWriter, r *http.Request) {
logRequest(r, 405, time.Since(start))
}
func handleHealthz(w http.ResponseWriter, r *http.Request) {
if _, err := os.Stat(uploadDir); err != nil {
log.Printf("Healthz check failed: %v", err)
http.Error(w, "Upload directory is not accessible", http.StatusServiceUnavailable)
return
}
w.WriteHeader(http.StatusOK)
}
func handleUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
filename := strings.TrimPrefix(r.URL.Path, "/")
if filename == "" {
@@ -126,8 +188,16 @@ func handleUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
}
defer file.Close()
written, err := io.Copy(file, r.Body)
limitedBody := http.MaxBytesReader(w, r.Body, maxUploadSize)
written, err := io.Copy(file, limitedBody)
if err != nil {
file.Close()
os.Remove(targetPath)
if isMaxBytesError(err) {
http.Error(w, fmt.Sprintf("File exceeds maximum upload size of %d bytes", maxUploadSize), http.StatusRequestEntityTooLarge)
logRequest(r, 413, time.Since(start))
return
}
http.Error(w, "Failed to write file", http.StatusInternalServerError)
logRequest(r, 500, time.Since(start))
return
@@ -144,16 +214,34 @@ func handleUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
logRequest(r, 200, time.Since(start))
}
func writeMultipartError(w http.ResponseWriter, status int, message string) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(status)
json.NewEncoder(w).Encode(map[string]string{"error": message})
}
func handleMultipartUpload(w http.ResponseWriter, r *http.Request, start time.Time) {
if err := r.ParseMultipartForm(32 << 20); err != nil {
http.Error(w, "Failed to parse multipart form", http.StatusBadRequest)
r.Body = http.MaxBytesReader(w, r.Body, maxUploadSize)
memoryLimit := maxUploadSize
if memoryLimit > maxMultipartMemory {
memoryLimit = maxMultipartMemory
}
if err := r.ParseMultipartForm(memoryLimit); err != nil {
if isMaxBytesError(err) {
writeMultipartError(w, http.StatusRequestEntityTooLarge, fmt.Sprintf("Upload exceeds maximum size of %d bytes", maxUploadSize))
logRequest(r, 413, time.Since(start))
return
}
writeMultipartError(w, http.StatusBadRequest, "Failed to parse multipart form")
logRequest(r, 400, time.Since(start))
return
}
files := r.MultipartForm.File["files"]
if len(files) == 0 {
http.Error(w, "No files provided", http.StatusBadRequest)
writeMultipartError(w, http.StatusBadRequest, "No files provided")
logRequest(r, 400, time.Since(start))
return
}
@@ -188,6 +276,7 @@ func handleMultipartUpload(w http.ResponseWriter, r *http.Request, start time.Ti
src.Close()
if err != nil {
os.Remove(targetPath)
continue
}
@@ -212,6 +301,12 @@ func handleDownload(w http.ResponseWriter, r *http.Request, start time.Time) {
id := parts[0]
filename := parts[1]
if !filepath.IsLocal(id) || !filepath.IsLocal(filename) {
http.NotFound(w, r)
logRequest(r, 404, time.Since(start))
return
}
filePath := filepath.Join(uploadDir, id, filename)
file, err := os.Open(filePath)
if err != nil {
@@ -219,12 +314,6 @@ func handleDownload(w http.ResponseWriter, r *http.Request, start time.Time) {
logRequest(r, 404, time.Since(start))
return
}
if !strings.HasPrefix(filePath, uploadDir) {
http.NotFound(w, r)
logRequest(r, 404, time.Since(start))
return
}
defer file.Close()
stat, err := file.Stat()
@@ -249,11 +338,10 @@ func generateUniqueID() (string, error) {
id := generateID()
targetDir := filepath.Join(uploadDir, id)
if _, err := os.Stat(targetDir); os.IsNotExist(err) {
if err := os.MkdirAll(targetDir, 0755); err != nil {
return "", err
}
if err := os.Mkdir(targetDir, 0755); err == nil {
return id, nil
} else if !os.IsExist(err) {
return "", err
}
}
@@ -324,6 +412,13 @@ func getProtocol(r *http.Request) string {
return "http"
}
// isMaxBytesError reports whether err was caused by exceeding the limit
// set on an http.MaxBytesReader.
func isMaxBytesError(err error) bool {
var maxBytesErr *http.MaxBytesError
return errors.As(err, &maxBytesErr)
}
func detectContentType(filename string) string {
ext := filepath.Ext(filename)
mimeType := mime.TypeByExtension(ext)
@@ -340,7 +435,11 @@ func logRequest(r *http.Request, status int, duration time.Duration) {
func serveFrontpage(w http.ResponseWriter, r *http.Request) {
proto := getProtocol(r)
host := getHost(r)
pageData := map[string]string{"Proto": proto, "Host": host}
pageData := map[string]any{
"Proto": proto,
"Host": host,
"MaxUploadSize": maxUploadSize,
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
indexHTML.Execute(w, pageData)
@@ -395,6 +494,9 @@ func cleanupProcedure(duration time.Duration, dryRun bool) {
if !d.IsDir() {
return nil
}
if filepath.Clean(path) == filepath.Clean(uploadDir) {
return nil
}
empty, err := isDirEmpty(path)
if empty {
emptyDirs = append(emptyDirs, path)
@@ -406,6 +508,8 @@ func cleanupProcedure(duration time.Duration, dryRun bool) {
}
for {
oldFiles = oldFiles[:0]
emptyDirs = emptyDirs[:0]
filepath.WalkDir(uploadDir, deleteIfTooOld)
for _, file := range oldFiles {