PostgreSQL Dump and Obfuscation Tool

We are excited to announce the release of Greenmask v0.1.0, marking the first production-ready version. This release addresses various bug fixes, introduces improvements, and includes documentation refactoring for enhanced clarity.

Greenmask overview

Key Features

• Cross-Platform Support: Developed in Go, Greenmask is free of platform dependencies.
• Type-Safe Database Operations: Ensures data validation and encoding, maintaining integrity.
• Transformation Validation: Guarantees correct and maintainable data transformations.
• Partitioned Table Support: Eases configuration for partitioned tables.
• Stateless and Backward Compatible: Compatible with standard PostgreSQL utilities.
• Parallel Execution: Boosts efficiency in dumping and restoration processes.
• Multiple Storage Options: Offers support for both local and remote storage solutions.

Use Cases

• Perfect for routine backup and restoration tasks.
• Aids in anonymization and data masking for staging environments and analytics.

Release notes

Improvements

• Improved verbosity in custom transformer interaction, accumulating stderr data and forwarding it in batches instead of writing it one by one.
• Added positional arguments for the list-transformers command, allowing specific transformer information retrieval (e.g., greenmask list-transformers RandomDate).
• Added version parameter --version that prints greenmask utility version
• Added Numeric parameters support for Int and Float transformers, with the caveat that in a few cases, it may lead to out-of-range errors.

Changes

• Updated dependencies to newer versions.
• Enhanced the stability of the JSON line interaction protocol by utilizing the stdlib JSON encoder/decoder.
• Modified the method for sending table metadata to custom transformers; now, it is sent via stdin in the first line in JSON format instead of providing it via command arguments.
• Refactored template functions naming.
• Rewritten NoiseDate transformer implementation for improved stability and predictability.
• Changed the default value for the Dict transformer fail_not_matched parameter to true.
• Rewritten Hash transformer, now providing a salt parameter and receiving a base64 encoded salt. If a salt is not provided, it generates one randomly.
• Added validator for NoiseDate and RandomDate transformers for the truncate parameter, issuing a warning if the provided value is invalid.
• Increased verbosity of parameter validation warnings, now properly forwarding warnings to stdout.

Fixes

• Resolved pgx driver connection leakage issue.
• Fixed deletion failure of dumps for S3 storage.
• Corrected cobra autocompletion for the Greenmask utility.
• Fixed NOT NULL constraint validation.
• Addressed JSON API interaction issues that previously caused deadlocks and timeouts.
• Fixed encode-decoding for binary parameters, ensuring accurate forwarding of values to custom transformers.
• Fixed the RandomChoice transformer to correctly marshal and unmarshal values during validation.
• Introduced the Nullable property for the SetNull transformer to enhance NOT NULL constraint validation.
• Resolved text wrapping issues for the validate command.
• Fixed build failures on Windows due to Linux platform dependencies.
• Corrected stdout readline buffer reading during interaction with custom transformers.
• Fixed integration tests.

Ecosystem Changes

• Implemented CI/CD for the entire project.
• Established a user-friendly playground in Docker-compose, including:
• Deployed Minio storage container.
• PostgreSQL container containing both the original database (Adventure Works) and the transformed (empty DB).
• Greenmask container itself.
• Refactored current readme files.

Community Participation

We highly value feedback and suggestions from the PostgreSQL community. Your insights and testing are crucial for enhancing Greenmask's robustness and utility.

Useful Links

• Documentation
• Release on GitHub
• Email Support