JSON, JSONL and Apache Avro Formats: Technical Questions and Answers
Engineering reference covering Technical Questions and Answers on RFC 8259 JSON trees, newline-delimited JSON streams, and binary Avro Object Container Files, how each format works at the byte level, how our deterministic generator builds and verifies them, and how to test them in QA pipelines.
Format hubs covered on this reference page
Part 1: Binary signatures, headers and specification rules
- 1. What leading byte signature or magic number identifies a JSON (.json) file at offset 0?
- When a binary inspector reads the start of a .json stream, it checks for UTF-8 text. In JSON files, this signature is part of IETF RFC 8259 UTF-8 text consisting of six structural tokens ({, }, [, ], :, ,), double-quoted strings with backslash escapes, IEEE 754 numbers, booleans (true, false), and null.
- 2. Which IANA MIME media type should an HTTP server return in Content-Type for .json downloads?
- Standardized delivery of JSON uses application/json. If a server sends a generic type without X-Content-Type-Options: nosniff, browsers may misinterpret .json payloads.
- 3. How does the internal structure of a JSON (.json) file organize headers and payload data?
- Inside .json, data is arranged as IETF RFC 8259 UTF-8 text consisting of six structural tokens ({, }, [, ], :, ,), double-quoted strings with backslash escapes, IEEE 754 numbers, booleans (true, false), and null. Parsers such as V8 JSON.parse, Python json, Jackson (Java), serde_json (Rust), jq, and SIMDJSON validate this layout before extracting application content.
- 4. What is the smallest valid byte size that the SampleTestFiles generator allows for .json?
- The minimum valid size for a generated JSON fixture on this site is 136 bytes (with a step increment of 1 byte), because a smaller buffer cannot hold the required .json header and trailer structures.
- 5. How does SampleTestFiles pad a .json file to reach an exact byte count like 10,240 bytes or 1,048,576 bytes?
- Instead of appending null bytes past the end of the file, our deterministic .json builder achieves exact byte targets by sizing the array of deterministic record objects and adjusting the final record's string field length so the closing bracket ] and newline hit the exact byte count.
- 6. What happens when a .json transfer is cut off halfway through uploading or downloading?
- In JSON streams, trailing commas after the last property or single-quoted strings violate RFC 8259 and throw SyntaxError in standard JSON.parse. Testing this behavior verifies that your ingestion worker returns a 400 or 422 validation error instead of crashing.
- 7. Which open-source libraries and desktop programs are commonly used to parse or inspect .json fixtures?
- Engineers typically validate and parse .json files using V8 JSON.parse, Python json, Jackson (Java), serde_json (Rust), jq, and SIMDJSON, each of which checks the specification rules defined by RFC 8259 and ECMA-404.
- 8. What are the primary software engineering and QA use cases for sample .json files?
- Teams use deterministic .json fixtures for REST API request/response validation, NoSQL document imports, configuration parsers, and large-payload memory profiling, ensuring both small 10 KB unit tests and large 100 MB stress tests behave predictably.
- 9. How does in-browser Web Worker generation work for .json files up to 32 MB?
- When you request a .json sample of 32 MB or less on /formats/json/, a background Web Worker runs our seeded generator off the main UI thread, builds the JSON blob in memory, verifies its SHA-256 hash via Web Crypto, and triggers a local download.
- 10. How does Service Worker streaming create 50 MB to 1 GB .json files without running out of browser RAM?
- For .json sizes above 32 MB on Chromium and Firefox, /sw.js intercepts the local /__gen/f/json/ request and emits 1 MB chunks through a ReadableStream directly to the browser download manager so the full file never resides in memory.
- 11. What is the published SHA-256 checksum for the 1kb (1,024-byte) .json sample file?
- The deterministic sample-json-1kb.json fixture (1,024 bytes) has the SHA-256 digest e6b79ec0b9d43a78e677d5f2dd9cb02e18011d5d7e0a0f55dbaad43add234c15. Every build produces those exact bytes.
- 12. What is the exact byte count and SHA-256 hash of sample-json-10kb.json?
- The file sample-json-10kb.json measures exactly 10,240 bytes and its SHA-256 checksum is f3272ec87fe1b8cf5702511bd42a3a33a809e1cf685b265add4ed11b7316a07f.
- 13. What is the exact byte length and SHA-256 digest of the 100kb .json fixture (sample-json-100kb.json)?
- The 100kb JSON fixture (sample-json-100kb.json) is 102,400 bytes long with SHA-256 hash 41dcab34af756a633fa486d862fdfb673116df9aeaa1ee459dc93a85b61e7e21.
- 14. How do I verify a downloaded .json sample on Linux, macOS, and Windows PowerShell?
- Run sha256sum sample-json-10kb.json on Linux, shasum -a 256 sample-json-10kb.json on macOS, or Get-FileHash sample-json-10kb.json -Algorithm SHA256 in Windows PowerShell, and compare the output against /formats/json/.
- 15. Why can a browser report the wrong file.type when a user selects a renamed .json file in an <input type='file'> control?
- Browsers populate File.type from the operating system extension mapping rather than reading the file's UTF-8 text signature. Dropping the file into /inspector/ or checking magic bytes on the backend prevents spoofed .json uploads.
- 16. How do I attach a sample .json file to an automated upload test in Playwright?
- Save the .json fixture from /formats/json/ into your test directory and call await page.locator('input[type="file"]').setInputFiles('fixtures/sample-json-10kb.json') before submitting the form.
- 17. How do I test .json file uploads in Cypress and Selenium WebDriver?
- In Cypress, run cy.get('input[type="file"]').selectFile('cypress/fixtures/sample-json-10kb.json'). In Selenium WebDriver, pass the absolute filesystem path of the .json sample to WebElement.sendKeys().
- 18. Does HTTP gzip or Brotli transfer compression shrink a .json file over the network?
- For JSON (structured data), network compression effectiveness depends on whether the payload is already entropy-coded; text-based formats compress heavily with Brotli, while pre-compressed binary streams gain almost nothing and waste CPU.
- 19. What happens if an API gateway or middleware reads a binary .json upload as a UTF-8 string?
- Decoding binary .json bytes as UTF-8 replaces any byte sequence invalid in UTF-8 with the 3-byte replacement character EF BF BD (U+FFFD), permanently corrupting IETF RFC 8259 UTF-8 text consisting of six structural tokens ({, }, [, ], :, ,), double-quoted strings with backslash escapes, IEEE 754 numbers, booleans (true, false), and null.
- 20. How does the in-browser preview on /sample-files/json/10kb/ display .json contents safely?
- When you click Preview file on a .json sample page, the file previewer generates or slices the local Blob in memory and renders text nodes or native media controls without executing untrusted markup.
- 21. Why are Safari and iOS browsers capped at a lower maximum in-memory size for .json than desktop Chrome or Firefox?
- WebKit on macOS and iOS does not hand streamed Service Worker responses to its download manager reliably, so the download controller assembles .json files in RAM within a safe device ceiling (64 MB on iOS, 256 MB on desktop Safari).
- 22. Do the sample .json files on SampleTestFiles contain any real personal data or third-party copyrighted material?
- No. Every .json file is synthesized deterministically by our own generator from mathematical patterns and invented records (using reserved example.com domains) under our CC0 public-domain licence (/licence/).
- 23. What specification or standard defines the .json format implemented by our generator?
- Our JSON generator follows RFC 8259 and ECMA-404, ensuring every header field and delimiter matches what V8 JSON.parse, Python json, Jackson (Java), serde_json (Rust), jq, and SIMDJSON expect.
- 24. How should a backend service store uploaded .json files in S3, object storage, or Google Cloud Storage?
- Validate the .json magic signature (UTF-8 text) on the first chunk, generate a UUID storage key instead of trusting the user filename, set Content-Type to application/json, and store the SHA-256 hex digest in object metadata.
- 25. Which size presets of .json are best for testing nginx client_max_body_size and AWS API Gateway limits?
- Use the 1 MB and 2 MB .json presets on /formats/json/ to test nginx's default 1m client_max_body_size, and use the 10 MB and 15 MB .json presets to test AWS API Gateway's 10 MB payload boundary.
Part 2: Internal container layout, padding and byte structure
- 1. What does the JSON (.json) specification table record for 'Extension'?
- For .json files in the Data category, the 'Extension' attribute is defined as: .json. Parsers rely on this property when validating JSON streams.
- 2. What does the JSON (.json) specification table record for 'MIME type'?
- For .json files in the Data category, the 'MIME type' attribute is defined as: application/json. Parsers rely on this property when validating JSON streams.
- 3. What does the JSON (.json) specification table record for 'Standard'?
- For .json files in the Data category, the 'Standard' attribute is defined as: RFC 8259 and ECMA-404. Parsers rely on this property when validating JSON streams.
- 4. What does the JSON (.json) specification table record for 'Encoding'?
- For .json files in the Data category, the 'Encoding' attribute is defined as: UTF-8 for interchange. Parsers rely on this property when validating JSON streams.
- 5. What does the JSON (.json) specification table record for 'Comments'?
- For .json files in the Data category, the 'Comments' attribute is defined as: Not allowed. Parsers rely on this property when validating JSON streams.
- 6. What does the JSON (.json) specification table record for 'Numbers'?
- For .json files in the Data category, the 'Numbers' attribute is defined as: Decimal; precision depends on the parser. Parsers rely on this property when validating JSON streams.
- 7. What is inside the 1kb JSON sample (sample-json-1kb.json) and what is its exact byte count?
- The 1kb .json fixture (sample-json-1kb.json) contains IETF RFC 8259 UTF-8 text consisting of six structural tokens ({, }, [, ], :, ,), double-quoted strings with backslash escapes, IEEE 754 numbers, booleans (true, false), and null. Its exact length is 1,024 bytes (1 1kb binary units) and its verified SHA-256 checksum is e6b79ec0b9d43a78e677d5f2dd9cb02e18011d5d7e0a0f55dbaad43add234c15.
- 8. What is inside the 5kb JSON sample (sample-json-5kb.json) and what is its exact byte count?
- The 5kb .json fixture (sample-json-5kb.json) contains IETF RFC 8259 UTF-8 text consisting of six structural tokens ({, }, [, ], :, ,), double-quoted strings with backslash escapes, IEEE 754 numbers, booleans (true, false), and null. Its exact length is 5,120 bytes (1 5kb binary units) and its verified SHA-256 checksum is acfefadcc50f3db10ac310ccfa21ab95abbac5e862a5effe554a79fc6d1f6348.
- 9. What is inside the 10kb JSON sample (sample-json-10kb.json) and what is its exact byte count?
- The 10kb .json fixture (sample-json-10kb.json) contains IETF RFC 8259 UTF-8 text consisting of six structural tokens ({, }, [, ], :, ,), double-quoted strings with backslash escapes, IEEE 754 numbers, booleans (true, false), and null. Its exact length is 10,240 bytes (1 10kb binary units) and its verified SHA-256 checksum is f3272ec87fe1b8cf5702511bd42a3a33a809e1cf685b265add4ed11b7316a07f.
- 10. What leading byte signature or magic number identifies a JSONL (.jsonl) file at offset 0?
- When a binary inspector reads the start of a .jsonl stream, it checks for UTF-8 text. In JSONL files, this signature is part of newline-delimited UTF-8 stream where each LF-terminated line is an independent, self-contained JSON object with no outer array wrapper.
- 11. Which IANA MIME media type should an HTTP server return in Content-Type for .jsonl downloads?
- Standardized delivery of JSONL uses application/jsonl. If a server sends a generic type without X-Content-Type-Options: nosniff, browsers may misinterpret .jsonl payloads.
- 12. How does the internal structure of a JSONL (.jsonl) file organize headers and payload data?
- Inside .jsonl, data is arranged as newline-delimited UTF-8 stream where each LF-terminated line is an independent, self-contained JSON object with no outer array wrapper. Parsers such as jq -c, Pandas read_json(lines=True), DuckDB, Vector, Fluent Bit, and Node.js readline + JSON.parse validate this layout before extracting application content.
- 13. What is the smallest valid byte size that the SampleTestFiles generator allows for .jsonl?
- The minimum valid size for a generated JSONL fixture on this site is 37 bytes (with a step increment of 1 byte), because a smaller buffer cannot hold the required .jsonl header and trailer structures.
- 14. How does SampleTestFiles pad a .jsonl file to reach an exact byte count like 10,240 bytes or 1,048,576 bytes?
- Instead of appending null bytes past the end of the file, our deterministic .jsonl builder achieves exact byte targets by emitting one compact JSON object per line until the exact byte budget is reached on the final newline.
- 15. What happens when a .jsonl transfer is cut off halfway through uploading or downloading?
- In JSONL streams, pretty-printing an object across multiple lines breaks line-by-line streaming parsers that call JSON.parse on each individual line. Testing this behavior verifies that your ingestion worker returns a 400 or 422 validation error instead of crashing.
- 16. Which open-source libraries and desktop programs are commonly used to parse or inspect .jsonl fixtures?
- Engineers typically validate and parse .jsonl files using jq -c, Pandas read_json(lines=True), DuckDB, Vector, Fluent Bit, and Node.js readline + JSON.parse, each of which checks the specification rules defined by Data.
- 17. What are the primary software engineering and QA use cases for sample .jsonl files?
- Teams use deterministic .jsonl fixtures for LLM fine-tuning datasets, BigQuery/Snowflake bulk streaming loads, log shipping pipelines, and constant-memory ETL workers, ensuring both small 10 KB unit tests and large 100 MB stress tests behave predictably.
- 18. How does in-browser Web Worker generation work for .jsonl files up to 32 MB?
- When you request a .jsonl sample of 32 MB or less on /formats/jsonl/, a background Web Worker runs our seeded generator off the main UI thread, builds the JSONL blob in memory, verifies its SHA-256 hash via Web Crypto, and triggers a local download.
- 19. How does Service Worker streaming create 50 MB to 1 GB .jsonl files without running out of browser RAM?
- For .jsonl sizes above 32 MB on Chromium and Firefox, /sw.js intercepts the local /__gen/f/jsonl/ request and emits 1 MB chunks through a ReadableStream directly to the browser download manager so the full file never resides in memory.
- 20. What is the published SHA-256 checksum for the 1kb (1,024-byte) .jsonl sample file?
- The deterministic sample-jsonl-1kb.jsonl fixture (1,024 bytes) has the SHA-256 digest c68974416de0b42c341067a23ff8f89cbd27ed525c2bbfb33d7977276d080c56. Every build produces those exact bytes.
- 21. What is the exact byte count and SHA-256 hash of sample-jsonl-100kb.jsonl?
- The file sample-jsonl-100kb.jsonl measures exactly 102,400 bytes and its SHA-256 checksum is 79d2aef233858e7f46f3dc9e2e6b814e22b33ed08641f6c5bb39e2f9081e2789.
- 22. What is the exact byte length and SHA-256 digest of the 10mb .jsonl fixture (sample-jsonl-10mb.jsonl)?
- The 10mb JSONL fixture (sample-jsonl-10mb.jsonl) is 10,485,760 bytes long with SHA-256 hash e6ac44daf233dfcafba54008362d230ed2730b53d0541b8cbc49301bc0477656.
- 23. How do I verify a downloaded .jsonl sample on Linux, macOS, and Windows PowerShell?
- Run sha256sum sample-jsonl-10kb.jsonl on Linux, shasum -a 256 sample-jsonl-10kb.jsonl on macOS, or Get-FileHash sample-jsonl-10kb.jsonl -Algorithm SHA256 in Windows PowerShell, and compare the output against /formats/jsonl/.
- 24. Why can a browser report the wrong file.type when a user selects a renamed .jsonl file in an <input type='file'> control?
- Browsers populate File.type from the operating system extension mapping rather than reading the file's UTF-8 text signature. Dropping the file into /inspector/ or checking magic bytes on the backend prevents spoofed .jsonl uploads.
- 25. How do I attach a sample .jsonl file to an automated upload test in Playwright?
- Save the .jsonl fixture from /formats/jsonl/ into your test directory and call await page.locator('input[type="file"]').setInputFiles('fixtures/sample-jsonl-10kb.jsonl') before submitting the form.
Part 3: In-browser generation, streaming and SHA-256 checksums
- 1. How do I test .jsonl file uploads in Cypress and Selenium WebDriver?
- In Cypress, run cy.get('input[type="file"]').selectFile('cypress/fixtures/sample-jsonl-10kb.jsonl'). In Selenium WebDriver, pass the absolute filesystem path of the .jsonl sample to WebElement.sendKeys().
- 2. Does HTTP gzip or Brotli transfer compression shrink a .jsonl file over the network?
- For JSONL (structured data), network compression effectiveness depends on whether the payload is already entropy-coded; text-based formats compress heavily with Brotli, while pre-compressed binary streams gain almost nothing and waste CPU.
- 3. What happens if an API gateway or middleware reads a binary .jsonl upload as a UTF-8 string?
- Decoding binary .jsonl bytes as UTF-8 replaces any byte sequence invalid in UTF-8 with the 3-byte replacement character EF BF BD (U+FFFD), permanently corrupting newline-delimited UTF-8 stream where each LF-terminated line is an independent, self-contained JSON object with no outer array wrapper.
- 4. How does the in-browser preview on /sample-files/jsonl/10kb/ display .jsonl contents safely?
- When you click Preview file on a .jsonl sample page, the file previewer generates or slices the local Blob in memory and renders text nodes or native media controls without executing untrusted markup.
- 5. Why are Safari and iOS browsers capped at a lower maximum in-memory size for .jsonl than desktop Chrome or Firefox?
- WebKit on macOS and iOS does not hand streamed Service Worker responses to its download manager reliably, so the download controller assembles .jsonl files in RAM within a safe device ceiling (64 MB on iOS, 256 MB on desktop Safari).
- 6. Do the sample .jsonl files on SampleTestFiles contain any real personal data or third-party copyrighted material?
- No. Every .jsonl file is synthesized deterministically by our own generator from mathematical patterns and invented records (using reserved example.com domains) under our CC0 public-domain licence (/licence/).
- 7. What specification or standard defines the .jsonl format implemented by our generator?
- Our JSONL generator follows the published JSONL specification, ensuring every header field and delimiter matches what jq -c, Pandas read_json(lines=True), DuckDB, Vector, Fluent Bit, and Node.js readline + JSON.parse expect.
- 8. How should a backend service store uploaded .jsonl files in S3, object storage, or Google Cloud Storage?
- Validate the .jsonl magic signature (UTF-8 text) on the first chunk, generate a UUID storage key instead of trusting the user filename, set Content-Type to application/jsonl, and store the SHA-256 hex digest in object metadata.
- 9. Which size presets of .jsonl are best for testing nginx client_max_body_size and AWS API Gateway limits?
- Use the 1 MB and 2 MB .jsonl presets on /formats/jsonl/ to test nginx's default 1m client_max_body_size, and use the 10 MB and 15 MB .jsonl presets to test AWS API Gateway's 10 MB payload boundary.
- 10. What does the JSONL (.jsonl) specification table record for 'Extension'?
- For .jsonl files in the Data category, the 'Extension' attribute is defined as: .jsonl. Parsers rely on this property when validating JSONL streams.
- 11. What does the JSONL (.jsonl) specification table record for 'MIME type'?
- For .jsonl files in the Data category, the 'MIME type' attribute is defined as: application/jsonl. Parsers rely on this property when validating JSONL streams.
- 12. What does the JSONL (.jsonl) specification table record for 'Also seen as'?
- For .jsonl files in the Data category, the 'Also seen as' attribute is defined as: application/x-ndjson. Parsers rely on this property when validating JSONL streams.
- 13. What does the JSONL (.jsonl) specification table record for 'Reference'?
- For .jsonl files in the Data category, the 'Reference' attribute is defined as: jsonlines.org. Parsers rely on this property when validating JSONL streams.
- 14. What does the JSONL (.jsonl) specification table record for 'Media type'?
- For .jsonl files in the Data category, the 'Media type' attribute is defined as: None is registered; application/jsonl and application/x-ndjson are both in use. Parsers rely on this property when validating JSONL streams.
- 15. What does the JSONL (.jsonl) specification table record for 'Record separator'?
- For .jsonl files in the Data category, the 'Record separator' attribute is defined as: Line feed. Parsers rely on this property when validating JSONL streams.
- 16. What does the JSONL (.jsonl) specification table record for 'Constraint'?
- For .jsonl files in the Data category, the 'Constraint' attribute is defined as: A record cannot contain a raw line break. Parsers rely on this property when validating JSONL streams.
- 17. What is inside the 1kb JSONL sample (sample-jsonl-1kb.jsonl) and what is its exact byte count?
- The 1kb .jsonl fixture (sample-jsonl-1kb.jsonl) contains newline-delimited UTF-8 stream where each LF-terminated line is an independent, self-contained JSON object with no outer array wrapper. Its exact length is 1,024 bytes (1 1kb binary units) and its verified SHA-256 checksum is c68974416de0b42c341067a23ff8f89cbd27ed525c2bbfb33d7977276d080c56.
- 18. What leading byte signature or magic number identifies a Avro (.avro) file at offset 0?
- When a binary inspector reads the start of a .avro stream, it checks for 4F 62 6A 01. In Avro files, this signature is part of an Avro Object Container File starting with magic bytes 4F 62 6A 01 ('Obj\x01'), a file header map containing the JSON schema ('avro.schema') and codec ('avro.codec'), and data blocks separated by 16-byte sync markers.
- 19. Which IANA MIME media type should an HTTP server return in Content-Type for .avro downloads?
- Standardized delivery of Avro uses application/avro. If a server sends a generic type without X-Content-Type-Options: nosniff, browsers may misinterpret .avro payloads.
- 20. How does the internal structure of a Avro (.avro) file organize headers and payload data?
- Inside .avro, data is arranged as an Avro Object Container File starting with magic bytes 4F 62 6A 01 ('Obj\x01'), a file header map containing the JSON schema ('avro.schema') and codec ('avro.codec'), and data blocks separated by 16-byte sync markers. Parsers such as Apache Avro (Java/C++/Python), fastavro, PySpark, Confluent Schema Registry tools, and DuckDB validate this layout before extracting application content.
- 21. What is the smallest valid byte size that the SampleTestFiles generator allows for .avro?
- The minimum valid size for a generated Avro fixture on this site is 757 bytes (with a step increment of 1 byte), because a smaller buffer cannot hold the required .avro header and trailer structures.
- 22. How does SampleTestFiles pad a .avro file to reach an exact byte count like 10,240 bytes or 1,048,576 bytes?
- Instead of appending null bytes past the end of the file, our deterministic .avro builder achieves exact byte targets by encoding variable-length ZigZag integers and UTF-8 string records across sync-marked data blocks to match the exact byte size.
- 23. What happens when a .avro transfer is cut off halfway through uploading or downloading?
- In Avro streams, altering the 16-byte sync marker at the end of a data block causes fastavro and Java Avro readers to raise an invalid sync marker exception. Testing this behavior verifies that your ingestion worker returns a 400 or 422 validation error instead of crashing.
- 24. Which open-source libraries and desktop programs are commonly used to parse or inspect .avro fixtures?
- Engineers typically validate and parse .avro files using Apache Avro (Java/C++/Python), fastavro, PySpark, Confluent Schema Registry tools, and DuckDB, each of which checks the specification rules defined by Apache Avro 1.11.
- 25. What are the primary software engineering and QA use cases for sample .avro files?
- Teams use deterministic .avro fixtures for Apache Kafka event schema evolution, Hadoop/Spark data lakes, and compact binary row serialization benchmarks, ensuring both small 10 KB unit tests and large 100 MB stress tests behave predictably.
Part 4: Parser libraries, edge cases and automated QA testing
- 1. How does in-browser Web Worker generation work for .avro files up to 32 MB?
- When you request a .avro sample of 32 MB or less on /formats/avro/, a background Web Worker runs our seeded generator off the main UI thread, builds the Avro blob in memory, verifies its SHA-256 hash via Web Crypto, and triggers a local download.
- 2. How does Service Worker streaming create 50 MB to 1 GB .avro files without running out of browser RAM?
- For .avro sizes above 32 MB on Chromium and Firefox, /sw.js intercepts the local /__gen/f/avro/ request and emits 1 MB chunks through a ReadableStream directly to the browser download manager so the full file never resides in memory.
- 3. What is the published SHA-256 checksum for the 1kb (1,024-byte) .avro sample file?
- The deterministic sample-avro-1kb.avro fixture (1,024 bytes) has the SHA-256 digest a53b9283c084120f422e5336ea75dac91b8b81e9bc57d9ab190f8392105f0f98. Every build produces those exact bytes.
- 4. What is the exact byte count and SHA-256 hash of sample-avro-100kb.avro?
- The file sample-avro-100kb.avro measures exactly 102,400 bytes and its SHA-256 checksum is 9a8550e91a21bca699c29dce54acfb615e6ec9f8e2ace4f15b4405990b130be2.
- 5. What is the exact byte length and SHA-256 digest of the 10mb .avro fixture (sample-avro-10mb.avro)?
- The 10mb Avro fixture (sample-avro-10mb.avro) is 10,485,760 bytes long with SHA-256 hash b7bb9375947e639f5ff6b22cf7c81fa95bf7f32e9bc5da10aa27661042e0f9c6.
- 6. How do I verify a downloaded .avro sample on Linux, macOS, and Windows PowerShell?
- Run sha256sum sample-avro-10kb.avro on Linux, shasum -a 256 sample-avro-10kb.avro on macOS, or Get-FileHash sample-avro-10kb.avro -Algorithm SHA256 in Windows PowerShell, and compare the output against /formats/avro/.
- 7. Why can a browser report the wrong file.type when a user selects a renamed .avro file in an <input type='file'> control?
- Browsers populate File.type from the operating system extension mapping rather than reading the file's 4F 62 6A 01 signature. Dropping the file into /inspector/ or checking magic bytes on the backend prevents spoofed .avro uploads.
- 8. How do I attach a sample .avro file to an automated upload test in Playwright?
- Save the .avro fixture from /formats/avro/ into your test directory and call await page.locator('input[type="file"]').setInputFiles('fixtures/sample-avro-10kb.avro') before submitting the form.
- 9. How do I test .avro file uploads in Cypress and Selenium WebDriver?
- In Cypress, run cy.get('input[type="file"]').selectFile('cypress/fixtures/sample-avro-10kb.avro'). In Selenium WebDriver, pass the absolute filesystem path of the .avro sample to WebElement.sendKeys().
- 10. Does HTTP gzip or Brotli transfer compression shrink a .avro file over the network?
- For Avro (structured data), network compression effectiveness depends on whether the payload is already entropy-coded; text-based formats compress heavily with Brotli, while pre-compressed binary streams gain almost nothing and waste CPU.
- 11. What happens if an API gateway or middleware reads a binary .avro upload as a UTF-8 string?
- Decoding binary .avro bytes as UTF-8 replaces any byte sequence invalid in UTF-8 with the 3-byte replacement character EF BF BD (U+FFFD), permanently corrupting an Avro Object Container File starting with magic bytes 4F 62 6A 01 ('Obj\x01'), a file header map containing the JSON schema ('avro.schema') and codec ('avro.codec'), and data blocks separated by 16-byte sync markers.
- 12. How does the in-browser preview on /sample-files/avro/10kb/ display .avro contents safely?
- When you click Preview file on a .avro sample page, the file previewer generates or slices the local Blob in memory and renders text nodes or native media controls without executing untrusted markup.
- 13. Why are Safari and iOS browsers capped at a lower maximum in-memory size for .avro than desktop Chrome or Firefox?
- WebKit on macOS and iOS does not hand streamed Service Worker responses to its download manager reliably, so the download controller assembles .avro files in RAM within a safe device ceiling (64 MB on iOS, 256 MB on desktop Safari).
- 14. Do the sample .avro files on SampleTestFiles contain any real personal data or third-party copyrighted material?
- No. Every .avro file is synthesized deterministically by our own generator from mathematical patterns and invented records (using reserved example.com domains) under our CC0 public-domain licence (/licence/).
- 15. What specification or standard defines the .avro format implemented by our generator?
- Our Avro generator follows Apache Avro 1.11, ensuring every header field and delimiter matches what Apache Avro (Java/C++/Python), fastavro, PySpark, Confluent Schema Registry tools, and DuckDB expect.
- 16. How should a backend service store uploaded .avro files in S3, object storage, or Google Cloud Storage?
- Validate the .avro magic signature (4F 62 6A 01) on the first chunk, generate a UUID storage key instead of trusting the user filename, set Content-Type to application/avro, and store the SHA-256 hex digest in object metadata.
- 17. Which size presets of .avro are best for testing nginx client_max_body_size and AWS API Gateway limits?
- Use the 1 MB and 2 MB .avro presets on /formats/avro/ to test nginx's default 1m client_max_body_size, and use the 10 MB and 15 MB .avro presets to test AWS API Gateway's 10 MB payload boundary.
- 18. What does the Avro (.avro) specification table record for 'Extension'?
- For .avro files in the Data category, the 'Extension' attribute is defined as: .avro. Parsers rely on this property when validating Avro streams.
- 19. What does the Avro (.avro) specification table record for 'MIME type'?
- For .avro files in the Data category, the 'MIME type' attribute is defined as: application/avro. Parsers rely on this property when validating Avro streams.
- 20. What does the Avro (.avro) specification table record for 'Also seen as'?
- For .avro files in the Data category, the 'Also seen as' attribute is defined as: avro/binary, application/octet-stream. Parsers rely on this property when validating Avro streams.
- 21. What does the Avro (.avro) specification table record for 'Specification'?
- For .avro files in the Data category, the 'Specification' attribute is defined as: Apache Avro 1.11. Parsers rely on this property when validating Avro streams.
- 22. What does the Avro (.avro) specification table record for 'Structure'?
- For .avro files in the Data category, the 'Structure' attribute is defined as: Header with schema and codec, then data blocks with sync markers. Parsers rely on this property when validating Avro streams.
- 23. What does the Avro (.avro) specification table record for 'Codecs'?
- For .avro files in the Data category, the 'Codecs' attribute is defined as: null and deflate required; snappy, zstandard, bzip2 and xz optional. Parsers rely on this property when validating Avro streams.
- 24. What does the Avro (.avro) specification table record for 'Integers'?
- For .avro files in the Data category, the 'Integers' attribute is defined as: Variable-length zigzag encoding. Parsers rely on this property when validating Avro streams.
- 25. What is inside the 1kb Avro sample (sample-avro-1kb.avro) and what is its exact byte count?
- The 1kb .avro fixture (sample-avro-1kb.avro) contains an Avro Object Container File starting with magic bytes 4F 62 6A 01 ('Obj\x01'), a file header map containing the JSON schema ('avro.schema') and codec ('avro.codec'), and data blocks separated by 16-byte sync markers. Its exact length is 1,024 bytes (1 1kb binary units) and its verified SHA-256 checksum is a53b9283c084120f422e5336ea75dac91b8b81e9bc57d9ab190f8392105f0f98.
Explore more topic-based technical format references
Browse all 20 deep-dive Q&A guides on Testing guides or Frequently asked questions.