Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 11 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
# Parquet Bulk Import for SQL Server
# Parquet to SQL Server Bulk Importer (.NET 8, Linux/Windows)

`ParquetBulkImporter` is a .NET 8 library for high-throughput imports of Parquet files into SQL Server tables.
[![CI](https://github.com/celoibarros/parquet-bulk-import-sqlserver/actions/workflows/ci.yml/badge.svg)](https://github.com/celoibarros/parquet-bulk-import-sqlserver/actions/workflows/ci.yml)

`ParquetBulkImporter` is a .NET 8 library to import Parquet files into Microsoft SQL Server using bulk copy patterns for high-throughput ETL and data ingestion workloads.

## Keywords

Parquet, SQL Server, Bulk Insert, SqlBulkCopy, ETL, Data Engineering, .NET, C#, Linux, Windows.

## Features

Expand All @@ -15,12 +21,12 @@
- .NET 8 SDK
- Python 3.10+ (for Python tests)
- SQL Server (local, container, or remote)
- ODBC Driver 18 for SQL Server (for connection string compatibility)
- ODBC Driver 18 for SQL Server

## Quick Start

```bash
git clone https://github.com/<your-username>/parquet-bulk-import-sqlserver.git
git clone https://github.com/celoibarros/parquet-bulk-import-sqlserver.git
cd parquet-bulk-import-sqlserver
dotnet build src/ParquetBulkImporter/ParquetBulkImporter.csproj -c Release
```
Expand All @@ -44,7 +50,7 @@ await importer.Execute();

## Test Configuration

Set these environment variables for integration tests:
Set environment variables for integration tests:

- `SQLSERVER_TEST_CONNECTION_STRING`
- `PARQUET_TEST_FOLDER` (optional, default: `tests/resources`)
Expand All @@ -70,8 +76,6 @@ python -m unittest discover -s tests/parquet_bulk_import_test -p "test_*.py"
docker compose up -d
```

The default SQL Server endpoint is `localhost,1433` with credentials defined in `docker-compose.yml`.

## Repository Layout

- `src/ParquetBulkImporter`: library source
Expand Down