dataframe and data connectivity#11
Merged
Merged
Conversation
Adds support for user-defined procedures with PRO/ENDPRO syntax, addressing the most critical missing feature in XDL. Changes: - Parser: Implement parameter and keyword parsing for procedure definitions - Parser: Add trailing comma handling for procedure calls - Parser: Add newline skipping at statement boundaries - Interpreter: Implement call_user_procedure() method with scope management - Interpreter: Add parameter binding and keyword argument handling - Add comprehensive test suite for procedure functionality Features: - Define procedures with parameters: PRO name, param1, param2 - Call procedures with arguments - Nested procedure calls - Local scope management - Parameter binding Known limitation: Procedures without parameters need at least one dummy parameter as a workaround for a parser edge case with trailing commas. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Add support for GOTO statements and labels in the XDL parser, enabling control flow jumps to labeled positions in the code. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Implements comprehensive SQLite integration for the XDL database connectivity module, enabling embedded database capabilities with both file-based and in-memory database support. Changes: - Add SQLite driver implementation with full CRUD operations - Support multiple connection string formats (sqlite://, file paths, :memory:) - Implement query execution with Recordset conversion - Add proper error handling and type conversion from SQLite to JSON - Include base64 encoding for blob data handling - Register SQLite in connection enum and module system - Add rusqlite dependency with bundled feature - Create comprehensive example demonstrating all SQLite features Features: - File-based and in-memory database support - Thread-safe connection handling with Mutex - Transaction support (BEGIN, COMMIT, ROLLBACK) - Index creation and management - Full SQL DDL/DML support (CREATE, INSERT, UPDATE, DELETE, SELECT) - Column metadata extraction 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Implements comprehensive DataFrame functionality for XDL with support for multiple data formats and seamless integration with ML, charting, and 3D visualization capabilities. Core Features: - DataFrame and Series data structures with labeled columns - Pandas/Spark-style operations (filter, select, groupby, sort, join) - Statistical operations (describe, sum, mean, count, value_counts) - Data manipulation (head, tail, column operations) Data Format Support: - CSV/TSV: Full read/write with type inference - Parquet: Columnar storage format for big data (requires feature flag) - Avro: Binary serialization format (requires feature flag) - Database: Seamless Recordset conversion from xdl-database File Structure: - src/dataframe.rs: Core DataFrame and GroupBy implementation - src/series.rs: Single column data structure - src/error.rs: Error handling types - src/readers/csv.rs: CSV/TSV reader with options - src/readers/parquet.rs: Parquet format support - src/readers/avro.rs: Avro format support - src/database.rs: Database Recordset integration Operations Implemented: - Selection: select(), column(), filter() - Aggregation: groupby().count(), groupby().mean(), groupby().sum() - Sorting: sort_by() - Inspection: head(), tail(), shape(), info(), describe() - Statistics: Series.mean(), Series.sum(), Series.unique(), Series.value_counts() - Export: to_json(), to_xdl_value(), write_csv(), write_tsv() Integration Points: - xdl-core: XdlValue and GdlArray compatibility - xdl-database: Recordset conversion for SQL queries - xdl-stdlib: ML functions, charting, and 3D visualization - Converts seamlessly between DataFrames and XDL arrays Examples Provided: 1. csv_analysis.xdl: Complete CSV workflow with filtering, grouping, plotting 2. parquet_example.xdl: Big data handling with Parquet format 3. database_integration.xdl: SQL database to DataFrame workflow with joins 4. ml_and_viz_integration.xdl: Full ML pipeline with classification and 3D viz Features: - default: CSV support + database integration - parquet-support: Enable Parquet format - avro-support: Enable Avro format - all: Enable all format support Dependencies: - csv 1.3 for CSV/TSV parsing - parquet 53 + arrow 53 for Parquet (optional) - apache-avro 0.17 for Avro (optional) - indexmap 2.0 for ordered column storage - Integration with xdl-database (optional) Testing: - 11 unit tests covering DataFrame, Series, and CSV operations - Type inference tests - All tests passing 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
dataframe and data connectivity