Import/export
CSV import and export the way django-import-export does it — including the part that makes it safe: import is two-phased. Nothing is written when a file is uploaded. First comes a dry run that diffs the file against what already exists and shows, row by row, what the import would do. Only when someone has looked at that preview and confirmed does anything execute.
@repo/import-export is that pattern as a building block: a pure planning engine,
RFC 4180 CSV handling, the preview table, and a browser download helper. No database,
no router, no i18n — the consumer owns persistence and every string.
The mapping from Django
| django-import-export | Here |
|---|---|
Resource (fields, semantics) | the options object you pass to planImport |
import_id_fields | keyColumn (+ optional normalizeKey) |
dry run → Result / RowResult | planImport() → ImportPlan / ImportRowPlan |
| the confirm screen | <ImportPreview /> + your confirm button |
confirm_form → real import | your code executes the confirmed plan |
| export action | toCsv() + downloadTextFile() |
The plan engine
Each CSV row is classified by what the import would do with it — new, update
(with the field-level diff), skip (exists, nothing differs) or error:
/** What the import would do with one CSV row. */
export type ImportRowKind = "new" | "update" | "skip" | "error";
/** Machine-readable row problems — the consumer translates them. */
export type ImportRowProblem = "missing-key" | "invalid" | "duplicate-in-file";
export interface ImportFieldChange {
column: string;
from: string;
to: string;
}
export interface ImportRowPlan {
/** 1-based data-row number (the header is not counted). */
line: number;
kind: ImportRowKind;
/** Row values keyed by (lower-cased) header column. */
values: Record<string, string>;
/** For `update`: the columns that differ, old → new. */
changes: ImportFieldChange[];
/** For `error`: why the row cannot be imported. */
problem?: ImportRowProblem;
}
export interface ImportPlan {
/** The (lower-cased) header columns, in file order. */
columns: string[];
rows: ImportRowPlan[];
counts: Record<ImportRowKind, number>;
/** Set when required header columns are missing — no rows are planned. */
missingColumns?: string[];
}The function itself is pure: you hand it the CSV and the existing records, so it
runs anywhere — in a server function against a table, or in the browser against a
Map (that is exactly what the demo below and the /sandbox/import-export harness
do):
/**
* Compute the import plan for a CSV: per row, decide `new` (no existing
* record has its key), `update` (exists and at least one update-column
* differs — with the field-level diff), `skip` (exists, nothing differs)
* or `error` (empty key, failed validation, or the same key appearing
* twice in the file).
*/
export function planImport(opts: {
csv: string;
/** Header columns that must exist for the file to make sense. */
requiredColumns: ReadonlyArray<string>;
/** django-import-export's `import_id_fields`: matches a row to an existing record. */
keyColumn: string;
/** Existing records, keyed by key-column value. */
existing: ReadonlyMap<string, Record<string, string>>;
/** Columns compared (and later written) on update. */
updateColumns: ReadonlyArray<string>;
/** Optional extra validation; `false` marks the row as an error. */
validateRow?: (values: Record<string, string>) => boolean;
/**
* Normalize the key before matching (e.g. lower-case emails). The
* `existing` map's keys must be normalized the same way.
*/
normalizeKey?: (key: string) => string;
}): ImportPlan {
const parsed = parseCsv(opts.csv);
const header = (parsed[0] ?? []).map((h) => h.trim().toLowerCase());
const counts: Record<ImportRowKind, number> = { new: 0, update: 0, skip: 0, error: 0 };
const missing = opts.requiredColumns.filter((c) => !header.includes(c));
if (missing.length > 0) {
return { columns: header, rows: [], counts, missingColumns: missing };
}
const seenKeys = new Set<string>();
// oxlint-disable-next-line oxc/no-map-spread -- bounded preview rows (one user-uploaded CSV), and the per-outcome `{ ...base }` returns read clearest; the early-return shape makes in-place mutation error-prone.
const rows = parsed.slice(1).map((cols, i): ImportRowPlan => {
const values: Record<string, string> = {};
for (const [col, name] of header.entries()) values[name] = (cols[col] ?? "").trim();
const base = { line: i + 1, values, changes: [] as ImportFieldChange[] };
const key = (opts.normalizeKey ?? ((k: string) => k))(values[opts.keyColumn] ?? "");
if (key === "") return { ...base, kind: "error", problem: "missing-key" };
if (seenKeys.has(key)) return { ...base, kind: "error", problem: "duplicate-in-file" };
seenKeys.add(key);
if (opts.validateRow && !opts.validateRow(values)) {
return { ...base, kind: "error", problem: "invalid" };
}
const current = opts.existing.get(key);
if (!current) return { ...base, kind: "new" };
const changes = opts.updateColumns
.filter((c) => values[c] !== undefined && values[c] !== (current[c] ?? ""))
.map((c) => ({ column: c, from: current[c] ?? "", to: values[c] ?? "" }));
return changes.length > 0 ? { ...base, kind: "update", changes } : { ...base, kind: "skip" };
});
for (const r of rows) counts[r.kind] += 1;
return { columns: header, rows, counts };
}Try it
The same engine and preview table the example's /admin/users uses, live against
two in-memory records. The sample CSV hits every outcome — try editing it:
CSV to import — edit it, the plan recomputes live
Existing records: Astrid Lindgren and Bo Nilsson
1 new · 1 updated · 1 skipped · 2 errors
| Action | name | |
|---|---|---|
| New | New Person | new@example.com |
| Skip | Astrid Lindgren | astrid@example.com |
| Update | bo@example.com | |
| Errormissing email | No Email | |
| Errorduplicate in file | Duplicate | new@example.com |
The two phases, wired to a real table
In the example app the consumer is the users admin. Both phases go through the same server-side planning helper — the client's copy of the plan is presentation only, never what gets executed:
Editing the CSV after a preview invalidates the plan in the UI — you preview again before you can confirm. The audit log records the executed counts.
CSV that round-trips
parseCsv/toCsv speak RFC 4180 — quoted fields, embedded commas, "" escapes,
newlines inside quotes, CRLF and a leading BOM. A file exported from Excel or
Google Sheets parses back unchanged:
/**
* Parse CSV text into rows of fields. Handles quoted fields (embedded
* commas, quotes and newlines), CRLF/LF line endings and a leading BOM.
* Fully empty lines are dropped.
*/
export function parseCsv(text: string): string[][] {
const src = text.codePointAt(0) === 0xfe_ff ? text.slice(1) : text;
const rows: string[][] = [];
let row: string[] = [];
let field = "";
let inQuotes = false;
const endField = () => {
row.push(field);
field = "";
};
const endRow = () => {
endField();
// A lone delimiter-less blank line parses as [""] — drop it.
if (row.length > 1 || row[0] !== "") rows.push(row);
row = [];
};
for (let i = 0; i < src.length; i += 1) {
const ch = src[i];
if (inQuotes) {
if (ch === '"') {
if (src[i + 1] === '"') {
field += '"';
i += 1;
} else {
inQuotes = false;
}
} else {
field += ch;
}
} else if (ch === '"' && field === "") {
inQuotes = true;
} else if (ch === ",") {
endField();
} else if (ch === "\n") {
endRow();
} else if (ch !== "\r") {
field += ch;
}
}
if (field !== "" || row.length > 0) endRow();
return rows;
}The export half is the same in every consumer: serialize, then hand the browser a
real file download. bom: true makes Excel read å/ä/ö correctly on double-click:
/** Trigger a download of `content` as `filename` in the current tab. */
export function downloadTextFile(opts: {
filename: string;
content: string;
/** Defaults to CSV. */
mimeType?: string;
/** Prepend a UTF-8 BOM so Excel reads å/ä/ö correctly on double-click. */
bom?: boolean;
}): void {
const parts = opts.bom ? [String.fromCodePoint(0xfe_ff), opts.content] : [opts.content];
const blob = new Blob(parts, { type: opts.mimeType ?? "text/csv;charset=utf-8" });
const url = URL.createObjectURL(blob);
const a = document.createElement("a");
a.href = url;
a.download = opts.filename;
document.body.append(a);
a.click();
a.remove();
URL.revokeObjectURL(url);
}Where to see it in the repo
/sandbox/import-export— the package in complete isolation: in-memory records, the plan recomputing live as you type. Spec:e2e/sandbox/import-export.spec.ts./admin/users— the real consumer with both phases against the user table. Spec:e2e/sandbox/users.spec.tsproves create and update through the preview.