fix(output): prevent CSV formula injection - #31
Conversation
| }; | ||
|
|
||
| const sanitize_csv_cell = (s: string): string=>{ | ||
| if (/^[=+\-@\t\r]/.test(s)) |
There was a problem hiding this comment.
There is a workaround for formula injection if there is a space before a formula like " =cmd|'/c calc'!A0" . Please detect dangerous prefixes after leading whitespace without removing that whitespace from the exported value. For example:
if (/^[\s\u00a0]*[=+\-@]/.test(s))
return "'" + s;
Please add tests for regular spaces, tabs, and NBSP.
This also changes numeric-looking string values such as "-100" and "+15". Is that intentional? If we want to preserve them, please define a strict numeric-string exemption and add tests. Otherwise, we should document that sanitization intentionally modifies all string cells starting with + or -.
| return JSON.stringify(data, null, 2); | ||
| }; | ||
|
|
||
| const print = (data: unknown, opts: Print_opts = {})=>{ |
There was a problem hiding this comment.
src/commands/dataset.ts -> handle_pipelines:
handle_pipelines() requests CSV from the API, so the result is already a string. Without -o, print() treats it as raw because format is not passed. With -o file.csv, serialize_csv() still returns the string unchanged before csv_escape.
Passing format to print() alone therefore won't fix the issue. We should choose:
- request structured JSON and serialize it locally when CSV output is requested, or
- parse the returned CSV and sanitize/re-serialize every cell.
Please add regression tests for both stdout redirection and -o file.csv
| return JSON.stringify(val); | ||
| }; | ||
|
|
||
| const sanitize_csv_cell = (s: string): string=>{ |
There was a problem hiding this comment.
Please add a global sanitize_csv configuration option that users can set with:
brightdata config set sanitize_csv false
brightdata config set sanitize_csv true
It should default to true. Since config values currently enter through the CLI as strings, please validate and convert "true"/"false" to actual booleans when setting or reading this option; "false" must not be treated as a truthy string. Other values should be rejected.
We can add per-command overrides later if needed (scrape, search, pipelines, discover, scraper, browser),
| const csv_escape = (val: unknown): string=>{ | ||
| const s = cell_to_string(val); | ||
| let s = cell_to_string(val); | ||
| if (typeof val == 'string' && get_config('sanitize_csv') !== false) |
There was a problem hiding this comment.
Please refactor it to not call get_config('sanitize_csv') for every cell (it reads config from the disk every time). It's better to read the config just once
| if (!sanitize || !csv) | ||
| return csv; | ||
| const rows = parse(csv, { | ||
| bom: true, |
There was a problem hiding this comment.
parse(..., {bom: true}) strips the input UTF-8 BOM, while stringify() does not restore it. This can regress Excel compatibility for pipeline CSV containing non-ASCII data. Please detect and preserve the BOM when re-serializing.
example:
const has_bom = csv.charCodeAt(0) == 0xFEFF;
return stringify(sanitized, {
bom: has_bom,
});
| return {...DEFAULTS}; | ||
| try { | ||
| const raw = fs.readFileSync(config_path, 'utf8'); | ||
| return {...DEFAULTS, ...JSON.parse(raw) as Config}; |
There was a problem hiding this comment.
The setter stores a boolean correctly, but load() still accepts "sanitize_csv": "false" as a string. Since "false" !== false, sanitization remains enabled, contrary to the configured value. Please normalize "true"/"false" while loading, or reject invalid non-boolean values, and add tests for persisted string values.
| const cleaned_result = format == 'json' | ||
| ? strip_nulls(result) | ||
| : format == 'csv' && typeof result == 'string' | ||
| ? sanitize_serialized_csv(result) | ||
| : result; |
There was a problem hiding this comment.
non-blocking:
Maybe replace it with if to make it more readable
| const cleaned_result = format == 'json' | |
| ? strip_nulls(result) | |
| : format == 'csv' && typeof result == 'string' | |
| ? sanitize_serialized_csv(result) | |
| : result; | |
| let cleaned_result = result; | |
| if (format == 'json') | |
| cleaned_result = strip_nulls(result); | |
| else if (format == 'csv' && typeof result == 'string') | |
| cleaned_result = sanitize_serialized_csv(result); |
| if (!sanitize || !csv) | ||
| return csv; | ||
| const rows = parse(csv, { | ||
| bom: true, |
There was a problem hiding this comment.
Please wrap parse in a try..catch and rethrow an actionable error. The error can explain how the user may explicitly opt out
let rows: string[][];
try {
rows = parse(csv, {...}) as string[][];
} catch (e) {
throw new Error(
'Failed to sanitize server CSV: '
+(e as Error).message
+'. Set sanitize_csv=false to export it without sanitization.'
);
}
| fail('sanitize_csv must be true or false'); | ||
| return; | ||
| } | ||
| set_config(valid_key, value == 'true'); |
There was a problem hiding this comment.
Please add regression tests for this behavior to guarantee:
- sanitize_csv false persists JSON boolean false, not string "false";
- sanitize_csv true persists JSON boolean true;
- an unsupported value such as sanitize_csv maybe is rejected and does not modify the config.
This is important because a persisted string "false" is truthy and would silently enable sanitization in code that performs a regular truthiness check.
Uh oh!
There was an error while loading. Please reload this page.