Character Encodings, UTF-8 BOM, Line Endings and Little-Endian vs. Big-Endian Byte Order Across 54 Formats: Questions and Answers
Engineering reference answering 100 distinct technical questions on handling UTF-8 vs. UTF-16 byte order marks (BOM), LF vs. CRLF line terminators, and little-endian vs. big-endian binary integers across all 54 file formats across all 54 file formats supported by SampleTestFiles.
Representative format hubs and diagnostic tools
Part 1: Document, spreadsheet and text format questions
- 1. How do character encoding, line endings, or integer byte endianness affect .avro (Avro) parsers?
- In .avro streams (Extension: .avro), an Avro Object Container File starting with magic bytes 4F 62 6A 01 ('Obj\x01') requires strict byte alignment; otherwise altering the 16-byte sync marker at the end of a data block causes fastavro and Java Avro readers to raise an invalid sync marker exception.
- 2. How do character encoding, line endings, or integer byte endianness affect .bin (BIN) parsers?
- In .bin streams (MIME type: application/octet-stream), high-entropy deterministic pseudo-random byte stream generated via a seeded 32-bit PRNG with no compressible repetition requires strict byte alignment; otherwise reading a binary .bin stream through a UTF-8 text decoder corrupts every byte >= 0x80 into U+FFFD replacement characters (EF BF BD).
- 3. How do character encoding, line endings, or integer byte endianness affect .bmp (BMP) parsers?
- In .bmp streams (Header: BITMAPFILEHEADER and a DIB header), 14-byte BITMAPFILEHEADER starting with 42 4D ('BM') requires strict byte alignment; otherwise forgetting that BMP rows are padded to a multiple of 4 bytes when width * 3 is not divisible by 4 skews every scanline diagonally.
- 4. How do character encoding, line endings, or integer byte endianness affect .css (CSS) parsers?
- In .css streams (Encoding: UTF-8), declaration blocks { property: value requires strict byte alignment; otherwise serving a .css file with Content-Type: text/plain when X-Content-Type-Options: nosniff is active causes browsers to refuse to apply the stylesheet.
- 5. How do character encoding, line endings, or integer byte endianness affect .csv (CSV) parsers?
- In .csv streams (Quoting: Double quotes, doubled to escape), fields delimited by commas (0x2C) requires strict byte alignment; otherwise an unclosed double quote on row N causes an RFC 4180 state machine to consume all subsequent rows into a single multi-line field until EOF.
- 6. How do character encoding, line endings, or integer byte endianness affect .docx (DOCX) parsers?
- In .docx streams (Macros: Not allowed in .docx; macro-enabled documents use .docm), _rels/.rels requires strict byte alignment; otherwise preventing OpenXML parsers from locating word/document.xml.
- 7. How do character encoding, line endings, or integer byte endianness affect .eml (EML) parsers?
- In .eml streams (Extension: .eml), Subject requires strict byte alignment; otherwise omitting the blank CRLF separator line between headers and body causes parsers to treat the entire message body as malformed header lines.
- 8. How do character encoding, line endings, or integer byte endianness affect .epub (EPUB) parsers?
- In .epub streams (MIME type: application/epub+zip), an OPF package document (.opf) requires strict byte alignment; otherwise placing META-INF/container.xml before the mimetype entry in the ZIP stream causes EPUBCheck error PKG-006 (mimetype must be first at offset 30).
- 9. How do character encoding, line endings, or integer byte endianness affect .flac (FLAC) parsers?
- In .flac streams (MIME type: audio/flac), channels requires strict byte alignment; otherwise modifying audio frames without updating the 16-byte raw-PCM MD5 checksum in the STREAMINFO block causes flac -t verification to fail.
- 10. How do character encoding, line endings, or integer byte endianness affect .geojson (GeoJSON) parsers?
- In .geojson streams (Coordinate order: Longitude, latitude, optional elevation), Polygon requires strict byte alignment; otherwise latitude] to [latitude.
- 11. How do character encoding, line endings, or integer byte endianness affect .gif (GIF) parsers?
- In .gif streams (Colour: Up to 256 colours per frame from a 24-bit palette), LZW-compressed image data sub-blocks requires strict byte alignment; otherwise missing the final 0x3B semicolon trailer byte or corrupting an LZW sub-block length byte causes frame decoders to fail.
- 12. How do character encoding, line endings, or integer byte endianness affect .gltf (glTF) parsers?
- In .gltf streams (Binary form: .glb, with the media type model/gltf-binary), 'bufferViews' requires strict byte alignment; otherwise miscalculating accessor byteOffset or byteLength against the decoded base64 buffer length triggers a BUFFER_VIEW_OUT_OF_BOUNDS error in glTF-Validator.
- 13. How do character encoding, line endings, or integer byte endianness affect .gpx (GPX) parsers?
- In .gpx streams (Coordinates: lat and lon attributes, decimal degrees, WGS 84), Topografix GPX 1.1 XML schema (<gpx version="1.1" creator="...">) containing waypoints (<wpt>) requires strict byte alignment; otherwise omitting the lat and lon attributes on <trkpt> or mixing local timezones without a Z UTC suffix in <time> breaks pace and elevation gain calculations.
- 14. How do character encoding, line endings, or integer byte endianness affect .gz (GZ) parsers?
- In .gz streams (Optional header fields: Original file name, comment, extra data), 32-bit mtime requires strict byte alignment; otherwise relying on the trailing 4-byte ISIZE field for files over 4 GB (since ISIZE wraps modulo 2^32).
- 15. How do character encoding, line endings, or integer byte endianness affect .html (HTML) parsers?
- In .html streams (Standard: HTML Living Standard (WHATWG)), and semantic <body> elements (<main> requires strict byte alignment; otherwise uploading unsanitized HTML to an endpoint that serves files without Content-Disposition: attachment allows stored XSS execution.
- 16. How do character encoding, line endings, or integer byte endianness affect .ico (ICO) parsers?
- In .ico streams (MIME type: image/vnd.microsoft.icon), byte size requires strict byte alignment; otherwise confusing type=1 (00 00 01 00 ICO icon) with type=2 (00 00 02 00 CUR cursor) in the 6-byte header.
- 17. How do character encoding, line endings, or integer byte endianness affect .ics (iCalendar) parsers?
- In .ics streams (Line folding: Lines longer than 75 octets continue on the next line after a space), VERSION:2.0 requires strict byte alignment; otherwise omitting mandatory UID or DTSTAMP properties inside a VEVENT block causes strict CalDAV servers and Outlook to reject the calendar import.
- 18. How do character encoding, line endings, or integer byte endianness affect .ini (INI) parsers?
- In .ini streams (Nesting: One level of sections), section headers enclosed in square brackets [section_name] requires strict byte alignment; otherwise placing key=value lines before the first [section] header causes Python's configparser to raise MissingSectionHeaderError.
- 19. How do character encoding, line endings, or integer byte endianness affect .jpg (JPG) parsers?
- In .jpg streams (Extension: .jpg), ITU-T T.81 / JFIF stream starting with SOI (FF D8) requires strict byte alignment; otherwise truncating the stream before the final FF D9 End-of-Image marker leaves the bottom scanlines grey or causes strict decoders to reject the image.
- 20. How do character encoding, line endings, or integer byte endianness affect .js (JavaScript) parsers?
- In .js streams (Module systems: ECMAScript modules and CommonJS), pure data arrays requires strict byte alignment; otherwise uploading .js files to a poorly configured static bucket without nosniff and attachment headers creates script-execution risks.
- 21. How do character encoding, line endings, or integer byte endianness affect .json (JSON) parsers?
- In .json streams (Standard: RFC 8259 and ECMA-404), } requires strict byte alignment; otherwise trailing commas after the last property or single-quoted strings violate RFC 8259 and throw SyntaxError in standard JSON.parse.
- 22. How do character encoding, line endings, or integer byte endianness affect .jsonl (JSONL) parsers?
- In .jsonl streams (Extension: .jsonl), newline-delimited UTF-8 stream where each LF-terminated line is an independent requires strict byte alignment; otherwise pretty-printing an object across multiple lines breaks line-by-line streaming parsers that call JSON.parse on each individual line.
- 23. How do character encoding, line endings, or integer byte endianness affect .kml (KML) parsers?
- In .kml streams (Coordinate order: Longitude, latitude, altitude), and <Polygon> elements with lon requires strict byte alignment; otherwise lat.
- 24. How do character encoding, line endings, or integer byte endianness affect .log (LOG) parsers?
- In .log streams (Growth: Append-only; rotated and compressed by tools such as logrotate), status codes requires strict byte alignment; otherwise multiline stack traces mixed into single-line access logs break regex line parsers unless multiline aggregation rules are configured.
- 25. How do character encoding, line endings, or integer byte endianness affect .m4a (M4A) parsers?
- In .m4a streams (Container: ISO base media file format (ISO/IEC 14496-12)), wrapping AAC-LC or Apple Lossless (ALAC) audio frames inside moov/mdat boxes with iTunes ilst metadata atoms requires strict byte alignment; otherwise confusing MP4-wrapped .m4a files (ftypM4A ) with raw ADTS .aac streams (FF F1).
Part 2: Image, audio, video, archive and 3D format questions
- 1. How do character encoding, line endings, or integer byte endianness affect .md (Markdown) parsers?
- In .md streams (MIME type: text/markdown), ordered and unordered lists requires strict byte alignment; otherwise leaving an unclosed triple-backtick code fence causes downstream sections to be swallowed into a single <pre><code> node.
- 2. How do character encoding, line endings, or integer byte endianness affect .mkv (MKV) parsers?
- In .mkv streams (Codecs: Almost any: H.264, HEVC, AV1, VP9, AAC, FLAC, Opus and more), and attached fonts requires strict byte alignment; otherwise assuming every 1A 45 DF A3 file is WebM without reading the EBML DocType string ('matroska' vs. 'webm').
- 3. How do character encoding, line endings, or integer byte endianness affect .mov (MOV) parsers?
- In .mov streams (Structure: Atoms: ftyp, moov, mdat), mdat requires strict byte alignment; otherwise uploading a QuickTime .mov recorded on iOS without checking whether the inner video track is H.264 (avc1) or HEVC (hvc1).
- 4. How do character encoding, line endings, or integer byte endianness affect .mp3 (MP3) parsers?
- In .mp3 streams (Frame length: 1,152 samples), an optional leading ID3v2 metadata tag (49 44 33 'ID3' with synchsafe 28-bit integer size) followed by MPEG-1 Audio Layer III frames starting with an 11-bit frame sync (FF FB) and a Xing/Info VBR header frame requires strict byte alignment; otherwise rejecting valid MP3 uploads because the file starts with 49 44 33 ('ID3') instead of a bare FF FB MPEG frame sync word.
- 5. How do character encoding, line endings, or integer byte endianness affect .mp4 (MP4) parsers?
- In .mp4 streams (Streaming: Supports HTTP range requests and fragmentation), video/audio tracks trak requires strict byte alignment; otherwise truncating an MP4 file whose moov box sits at the end of the file leaves the mdat frames unindexed and unplayable.
- 6. How do character encoding, line endings, or integer byte endianness affect .obj (OBJ) parsers?
- In .obj streams (Extension: .obj), texture coordinates (vt u v) requires strict byte alignment; otherwise using 0-based indices instead of Wavefront's mandatory 1-based vertex indexing shifts every triangle face or crashes the mesh loader.
- 7. How do character encoding, line endings, or integer byte endianness affect .ods (ODS) parsers?
- In .ods streams (MIME type: application/vnd.oasis.opendocument.spreadsheet), an OASIS OpenDocument Spreadsheet ZIP archive with an uncompressed mimetype entry at offset 30 and <table:table-row>/<table:table-cell> nodes in content.xml requires strict byte alignment; otherwise omitting office:value-type='float' on numeric cells causes formulas to treat cell contents as zero-valued text strings.
- 8. How do character encoding, line endings, or integer byte endianness affect .odt (ODT) parsers?
- In .odt streams (Standard: OASIS OpenDocument 1.3; ISO/IEC 26300), and META-INF/manifest.xml requires strict byte alignment; otherwise compressing the first mimetype entry with DEFLATE violates the OASIS ODF packaging rule requiring magic-byte sniffers to read the MIME string at offset 38 in plain ASCII.
- 9. How do character encoding, line endings, or integer byte endianness affect .pdf (PDF) parsers?
- In .pdf streams (Structure: Header, body objects, cross-reference table, trailer), numbered indirect objects (N 0 obj ... endobj) requires strict byte alignment; otherwise forcing viewers to attempt a full linear object scan.
- 10. How do character encoding, line endings, or integer byte endianness affect .png (PNG) parsers?
- In .png streams (Colour: Greyscale, palette or truecolour, 1 to 16 bits per channel), zlib/DEFLATE-compressed IDAT pixel scanlines with filter bytes requires strict byte alignment; otherwise altering a single byte in IHDR or tEXt without updating its 4-byte big-endian CRC-32 causes libpng and browsers to abort decoding with a CRC error.
- 11. How do character encoding, line endings, or integer byte endianness affect .pptx (PPTX) parsers?
- In .pptx streams (Macros: Not allowed in .pptx; macro-enabled files use .pptm), and DrawingML vector/text shapes measured in English Metric Units (1 inch = 914 requires strict byte alignment; otherwise missing relationship entries in ppt/slides/_rels/slide1.xml.rels breaks embedded image and layout references when opened in PowerPoint.
- 12. How do character encoding, line endings, or integer byte endianness affect .rtf (RTF) parsers?
- In .rtf streams (MIME type: application/rtf), \par) requires strict byte alignment; otherwise dropping the final closing curly brace } leaves the root \rtf1 group unclosed.
- 13. How do character encoding, line endings, or integer byte endianness affect .sql (SQL) parsers?
- In .sql streams (MIME type: application/sql), INSERT INTO ... VALUES ... requires strict byte alignment; otherwise unescaped single quotes inside a SQL string literal break statement parsing and expose SQL injection vulnerabilities in naive string builders.
- 14. How do character encoding, line endings, or integer byte endianness affect .srt (SRT) parsers?
- In .srt streams (Specification: None; a de facto standard), and a blank separator line requires strict byte alignment; otherwise using a period instead of a comma before milliseconds (00:00:01.500 instead of 00:00:01.
- 15. How do character encoding, line endings, or integer byte endianness affect .stl (STL) parsers?
- In .stl streams (Encodings: ASCII (text) and binary), either an 80-byte binary header + 4-byte uint32 triangle count + N * 50-byte triangle records (12-byte normal vector + three 12-byte float32 vertices + 2-byte attribute count) or an ASCII 'solid ... facet normal ... endsolid' mesh requires strict byte alignment; otherwise when many binary STL exporters write 'solid' in their 80-byte binary header (always check if 84 + uint32_at_80 * 50 == file_size).
- 16. How do character encoding, line endings, or integer byte endianness affect .svg (SVG) parsers?
- In .svg streams (Compressed form: .svgz (gzip)), <g> requires strict byte alignment; otherwise omitting the xmlns="http://www.w3.org/2000/svg" namespace attribute causes browsers opening the file directly to display raw XML text instead of rendering the graphic.
- 17. How do character encoding, line endings, or integer byte endianness affect .tar (TAR) parsers?
- In .tar streams (End marker: Two blocks of zero bytes), sequential 512-byte blocks where each archived file is preceded by a 512-byte POSIX ustar header (magic 'ustar' 75 73 74 61 72 at byte offset 257 requires strict byte alignment; otherwise or failing to validate octal header checksums.
- 18. How do character encoding, line endings, or integer byte endianness affect .tiff (TIFF) parsers?
- In .tiff streams (Extension: .tiff), Length requires strict byte alignment; otherwise hardcoding little-endian integer reads in a parser causes it to fail on big-endian (4D 4D 00 2A) TIFF files produced by macOS and high-end scanners.
- 19. How do character encoding, line endings, or integer byte endianness affect .toml (TOML) parsers?
- In .toml streams (MIME type: application/toml), strongly typed arrays requires strict byte alignment; otherwise redefining the same [table] header twice or mixing data types in an inline array (in TOML v0.4 parsers) fails validation.
- 20. How do character encoding, line endings, or integer byte endianness affect .tsv (TSV) parsers?
- In .tsv streams (Media type registration: IANA text/tab-separated-values (1993)), IANA text/tab-separated-values tabular text where columns are separated by horizontal tab characters (0x09) and rows by LF or CRLF requires strict byte alignment; otherwise untrimmed literal tab characters inside a free-text field shift all subsequent column values one column to the right.
- 21. How do character encoding, line endings, or integer byte endianness affect .txt (TXT) parsers?
- In .txt streams (Line endings: LF on Unix and macOS, CRLF on Windows), raw unformatted UTF-8 character sequences terminated by LF (0x0A) or CRLF (0x0D 0x0A) newline bytes with no binary header requires strict byte alignment; otherwise injecting invalid UTF-8 continuation bytes (0xFF.
- 22. How do character encoding, line endings, or integer byte endianness affect .vtt (WebVTT) parsers?
- In .vtt streams (Timestamp: HH:MM:SS.mmm with a full stop before milliseconds), followed by a blank line and timed cues using period millisecond separators (HH:MM:SS.mmm --> HH:MM:SS.mmm) and optional cue positioning settings requires strict byte alignment; otherwise missing the leading WEBVTT header on line 1 or prepending a non-UTF-8 encoding causes HTML5 <track> elements to discard the entire subtitle track silently.
- 23. How do character encoding, line endings, or integer byte endianness affect .wav (WAV) parsers?
- In .wav streams (Size limit: 4 GB, because sizes are 32-bit), sample rate requires strict byte alignment; otherwise 295 bytes overflows the unsigned 32-bit RIFF chunk size field (requiring RF64 or Sony Wave64 .w64).
- 24. How do character encoding, line endings, or integer byte endianness affect .webm (WebM) parsers?
- In .webm streams (Extension: .webm), followed by a Segment containing Info requires strict byte alignment; otherwise serving a .webm file with an incompatible audio codec (such as AAC instead of Vorbis/Opus) violates the WebM container specification.
- 25. How do character encoding, line endings, or integer byte endianness affect .webp (WebP) parsers?
- In .webp streams (MIME type: image/webp), VP8L (lossless) requires strict byte alignment; otherwise leaving the 32-bit RIFF payload size at bytes 4–7 inconsistent with the actual file length causes strict RIFF demuxers to reject the file.
- 26. How do character encoding, line endings, or integer byte endianness affect .xlsx (XLSX) parsers?
- In .xlsx streams (Standard: ECMA-376 and ISO/IEC 29500), xl/worksheets/sheet1.xml (<row> and <c> cell nodes) requires strict byte alignment; otherwise mismatching a cell's t='s' shared-string index against the count of <si> items in xl/sharedStrings.xml causes spreadsheet readers to throw an index-out-of-bounds error.
- 27. How do character encoding, line endings, or integer byte endianness affect .xml (XML) parsers?
- In .xml streams (Also seen as: text/xml), and entity escapes (& requires strict byte alignment; otherwise including an unescaped ampersand (&) or left angle bracket (<) inside element text immediately halts W3C-compliant XML parsers with a fatal well-formedness error.
- 28. How do character encoding, line endings, or integer byte endianness affect .yaml (YAML) parsers?
- In .yaml streams (Indentation: Spaces only; tabs are not allowed), and # comments requires strict byte alignment; otherwise mixing tab characters (0x09) with spaces in indentation or leaving colons unquoted inside strings triggers a YAML scanner error.
- 29. How do character encoding, line endings, or integer byte endianness affect .zip (ZIP) parsers?
- In .zip streams (Compression: Deflate or none per entry; others are optional), PKWARE APPNOTE container consisting of Local File Headers (50 4B 03 04) requires strict byte alignment; otherwise corrupting payload bytes so the computed CRC-32 disagrees with the header (tested by our zip-corrupted-crc fixture) or Zip Slip '../' path traversal.
Part 3: Architecture, implementation and pipeline questions
- 1. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .avro file?
- For binary .avro files, prepending EF BB BF shifts the 4F 62 6A 01 signature by 3 bytes and invalidates the header; for text .avro files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 2. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .bin file?
- For binary .bin files, prepending EF BB BF shifts the None signature by 3 bytes and invalidates the header; for text .bin files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 3. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .bmp file?
- For binary .bmp files, prepending EF BB BF shifts the 42 4D signature by 3 bytes and invalidates the header; for text .bmp files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 4. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .css file?
- For binary .css files, prepending EF BB BF shifts the UTF-8 text signature by 3 bytes and invalidates the header; for text .css files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 5. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .csv file?
- For binary .csv files, prepending EF BB BF shifts the UTF-8 text signature by 3 bytes and invalidates the header; for text .csv files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 6. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .docx file?
- For binary .docx files, prepending EF BB BF shifts the 50 4B 03 04 signature by 3 bytes and invalidates the header; for text .docx files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 7. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .eml file?
- For binary .eml files, prepending EF BB BF shifts the UTF-8 text signature by 3 bytes and invalidates the header; for text .eml files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 8. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .epub file?
- For binary .epub files, prepending EF BB BF shifts the 50 4B 03 04 signature by 3 bytes and invalidates the header; for text .epub files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 9. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .flac file?
- For binary .flac files, prepending EF BB BF shifts the 66 4C 61 43 signature by 3 bytes and invalidates the header; for text .flac files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 10. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .geojson file?
- For binary .geojson files, prepending EF BB BF shifts the UTF-8 text signature by 3 bytes and invalidates the header; for text .geojson files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 11. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .gif file?
- For binary .gif files, prepending EF BB BF shifts the 47 49 46 38 39 61 signature by 3 bytes and invalidates the header; for text .gif files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 12. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .gltf file?
- For binary .gltf files, prepending EF BB BF shifts the UTF-8 text signature by 3 bytes and invalidates the header; for text .gltf files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 13. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .gpx file?
- For binary .gpx files, prepending EF BB BF shifts the UTF-8 text signature by 3 bytes and invalidates the header; for text .gpx files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 14. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .gz file?
- For binary .gz files, prepending EF BB BF shifts the 1F 8B 08 signature by 3 bytes and invalidates the header; for text .gz files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 15. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .html file?
- For binary .html files, prepending EF BB BF shifts the 3C 21 44 4F 43 54 59 50 45 signature by 3 bytes and invalidates the header; for text .html files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 16. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .ico file?
- For binary .ico files, prepending EF BB BF shifts the 00 00 01 00 signature by 3 bytes and invalidates the header; for text .ico files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 17. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .ics file?
- For binary .ics files, prepending EF BB BF shifts the UTF-8 text signature by 3 bytes and invalidates the header; for text .ics files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 18. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .ini file?
- For binary .ini files, prepending EF BB BF shifts the UTF-8 text signature by 3 bytes and invalidates the header; for text .ini files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 19. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .jpg file?
- For binary .jpg files, prepending EF BB BF shifts the FF D8 FF signature by 3 bytes and invalidates the header; for text .jpg files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 20. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .js file?
- For binary .js files, prepending EF BB BF shifts the UTF-8 text signature by 3 bytes and invalidates the header; for text .js files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 21. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .json file?
- For binary .json files, prepending EF BB BF shifts the UTF-8 text signature by 3 bytes and invalidates the header; for text .json files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 22. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .jsonl file?
- For binary .jsonl files, prepending EF BB BF shifts the UTF-8 text signature by 3 bytes and invalidates the header; for text .jsonl files, some parsers strip EF BB BF while strict lexers may reject byte 0.
- 23. What happens if a 3-byte UTF-8 Byte Order Mark (EF BB BF) is prepended to a .kml file?
- For binary .kml files, prepending EF BB BF shifts the UTF-8 text signature by 3 bytes and invalidates the header; for text .kml files, some parsers strip EF BB BF while strict lexers may reject byte 0.
Part 4: Verification, edge cases and automation questions
- 1. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .avro?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after 4F 62 6A 01, and changes the SHA-256 hash from a53b9283c084120f…; always set * binary in .gitattributes for test fixtures.
- 2. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .bin?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after None, and changes the SHA-256 hash from bb174eca55fcab7b…; always set * binary in .gitattributes for test fixtures.
- 3. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .bmp?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after 42 4D, and changes the SHA-256 hash from d7833ceeb6df6a3f…; always set * binary in .gitattributes for test fixtures.
- 4. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .css?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after UTF-8 text, and changes the SHA-256 hash from d66dd341e097bfbf…; always set * binary in .gitattributes for test fixtures.
- 5. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .csv?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after UTF-8 text, and changes the SHA-256 hash from cdddf7fab2967f74…; always set * binary in .gitattributes for test fixtures.
- 6. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .docx?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after 50 4B 03 04, and changes the SHA-256 hash from f035af1f12c5f378…; always set * binary in .gitattributes for test fixtures.
- 7. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .eml?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after UTF-8 text, and changes the SHA-256 hash from fcb83c22ab1b25f4…; always set * binary in .gitattributes for test fixtures.
- 8. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .epub?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after 50 4B 03 04, and changes the SHA-256 hash from 7aeb3a14b30fc59f…; always set * binary in .gitattributes for test fixtures.
- 9. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .flac?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after 66 4C 61 43, and changes the SHA-256 hash from 7d5b1004d60d26ed…; always set * binary in .gitattributes for test fixtures.
- 10. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .geojson?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after UTF-8 text, and changes the SHA-256 hash from 9be0172ddbe0d3f7…; always set * binary in .gitattributes for test fixtures.
- 11. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .gif?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after 47 49 46 38 39 61, and changes the SHA-256 hash from 66c85de4a8d004e4…; always set * binary in .gitattributes for test fixtures.
- 12. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .gltf?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after UTF-8 text, and changes the SHA-256 hash from 00005d45316577b7…; always set * binary in .gitattributes for test fixtures.
- 13. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .gpx?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after UTF-8 text, and changes the SHA-256 hash from 753bd5be3339a3bc…; always set * binary in .gitattributes for test fixtures.
- 14. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .gz?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after 1F 8B 08, and changes the SHA-256 hash from d67bc6e2b98c4e46…; always set * binary in .gitattributes for test fixtures.
- 15. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .html?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after 3C 21 44 4F 43 54 59 50 45, and changes the SHA-256 hash from eabca5a92fca425f…; always set * binary in .gitattributes for test fixtures.
- 16. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .ico?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after 00 00 01 00, and changes the SHA-256 hash from 144845a0b3fecb0e…; always set * binary in .gitattributes for test fixtures.
- 17. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .ics?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after UTF-8 text, and changes the SHA-256 hash from b53dd9960c400e35…; always set * binary in .gitattributes for test fixtures.
- 18. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .ini?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after UTF-8 text, and changes the SHA-256 hash from 58c71755af8a90ba…; always set * binary in .gitattributes for test fixtures.
- 19. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .jpg?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after FF D8 FF, and changes the SHA-256 hash from 8ee87ac7e59dde63…; always set * binary in .gitattributes for test fixtures.
- 20. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .js?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after UTF-8 text, and changes the SHA-256 hash from 3555814484c57fa1…; always set * binary in .gitattributes for test fixtures.
- 21. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .json?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after UTF-8 text, and changes the SHA-256 hash from e6b79ec0b9d43a78…; always set * binary in .gitattributes for test fixtures.
- 22. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .jsonl?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after UTF-8 text, and changes the SHA-256 hash from c68974416de0b42c…; always set * binary in .gitattributes for test fixtures.
- 23. Why does converting LF (0x0A) to CRLF (0x0D 0x0A) on Windows Git checkouts break binary and checksum tests for .kml?
- Replacing 0x0A with 0x0D 0x0A alters the byte length, invalidates byte offsets after UTF-8 text, and changes the SHA-256 hash from 405296fc546173a0…; always set * binary in .gitattributes for test fixtures.
Explore all 40 topic-based technical format references
Browse the complete engineering reference library on Testing guides or Frequently asked questions.