fix: sanitize Avro field names on write, respect iceberg-field-name on read

[SreeramGarlapati]

Summary

Iceberg field names can be anything (123column, field.with.dots, etc.) but Avro requires names to match [A-Za-z_][A-Za-z0-9_]*. Java handles this with a sanitize-on-write + restore-on-read protocol using the iceberg-field-name custom Avro property. iceberg-rust was doing neither — writing invalid names directly and ignoring the property on read.

This causes two interop failures:

1. Write path: iceberg-rust produces invalid Avro when field names have leading digits or special chars → strict parsers (Java, Python) reject the file
2. Read path: Avro files written by Java with sanitized names get the wrong field names when read by iceberg-rust

Changes

Write path (Iceberg→Avro conversion in SchemaToAvroSchema::field):

• Added is_valid_avro_name() — checks [A-Za-z_][A-Za-z0-9_]*
• Added sanitize_avro_name() — matches Java's AvroSchemaUtil.sanitize():
  • Leading digit: prefix _ (e.g., 123col → _123col)
  • Special chars: _x<HEX> (e.g., field.name → field_x2Ename)
  • Operates on UTF-16 code units to match Java's charAt() behavior for supplementary chars
• When sanitization is needed: stores original name in iceberg-field-name property

Read path (Avro→Iceberg conversion in AvroSchemaToSchema::record):

• Checks iceberg-field-name custom attribute first, falls back to Avro field name

Java reference

• AvroSchemaUtil.sanitize()
• TypeToSchema.struct() — stores ICEBERG_FIELD_NAME_PROP
• AvroSchemaUtil.ICEBERG_FIELD_NAME_PROP

Closes #2535

Test plan

• test_is_valid_avro_name — validates detection of invalid names
• test_sanitize_avro_name — ASCII edge cases match Java behavior
• test_sanitize_avro_name_unicode — BMP and supplementary char handling (surrogate pairs)
• test_sanitization_round_trip — Iceberg→Avro→Iceberg preserves original names
• test_avro_to_iceberg_uses_iceberg_field_name_property — reads Java-written schemas correctly
• All 1301 existing tests pass (no regressions in bidirectional schema conversion tests)

[xanderbailey]

Nice PR, thanks for the contribution, left a couple of questions

[xanderbailey]

It would be nice if we could use something from https://github.com/apache/avro-rs/blob/4edb1ce1ae1ab5bd3fafb08ca3f622946c01c9fd/avro/src/validator.rs#L4 but it looks like validate_record_field_name is pub(crate). Is it work filing an issue upstream to see if we could expose something that would allow us to validate against their implementation?

[xanderbailey]

    /**
     * Determines if the specified character is a digit.
     * <p>
     * A character is a digit if its general category type, provided
     * by {@code Character.getType(ch)}, is
     * {@code DECIMAL_DIGIT_NUMBER}.
     * <p>
     * Some Unicode character ranges that contain digits:
     * <ul>
     * <li>{@code '\u005Cu0030'} through {@code '\u005Cu0039'},
     *     ISO-LATIN-1 digits ({@code '0'} through {@code '9'})
     * <li>{@code '\u005Cu0660'} through {@code '\u005Cu0669'},
     *     Arabic-Indic digits
     * <li>{@code '\u005Cu06F0'} through {@code '\u005Cu06F9'},
     *     Extended Arabic-Indic digits
     * <li>{@code '\u005Cu0966'} through {@code '\u005Cu096F'},
     *     Devanagari digits
     * <li>{@code '\u005CuFF10'} through {@code '\u005CuFF19'},
     *     Fullwidth digits
     * </ul>
     *
     * Many other character ranges contain digits as well.
     *
     * <p><b>Note:</b> This method cannot handle <a
     * href="#supplementary"> supplementary characters</a>. To support
     * all Unicode characters, including supplementary characters, use
     * the {@link #isDigit(int)} method.
     *
     * @param   ch   the character to be tested.
     * @return  {@code true} if the character is a digit;
     *          {@code false} otherwise.
     * @see     Character#digit(char, int)
     * @see     Character#forDigit(int, int)
     * @see     Character#getType(char)
     */
    public static boolean isDigit(char ch) {
        return isDigit((int)ch);
    }

https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/avro/AvroSchemaUtil.java#L551
Java's isDigit() actually covers more than just ascii digits. This is a pretty niche edge-case.

Trying to reason about this in my head and I don't think it matters for interop that the representations are identical because both will correctly underscore the field and restore it from the map? Can you check my reasoning here?