Harness Liquibase MongoDB Extension

This is a Harness-enhanced fork of the Liquibase MongoDB Extension with additional features and improvements.

Maven Repository: Harness Maven Public Repository

Documentation: Harness Database DevOps - MongoDB Commands

Table of contents

1. Introduction
2. Release Notes
3. Implemented Changes
4. Connection String Formats
5. Getting Started
6. Running tests
7. Integration
8. Harness Enhancements
9. Contributing
10. License

Introduction

This is a Harness-enhanced Liquibase extension for MongoDB support, forked from the Liquibase MongoDB Extension.

The original extension resulted as an alternative to existing MongoDB evolution tools that were basically wrappers over the deprecated db.eval shell method (deprecated starting from MongoDB 4.2).

Harness has enhanced this extension with additional features including:

• Mongo Native Executor - Execute native MongoDB operations with improved performance
• Enhanced database change management capabilities
• Additional operational improvements and optimizations

The extension allows calling specific mongo-java-driver methods through Liquibase's changeset framework, providing enterprise-grade database change management for MongoDB deployments.

Release Notes

Harness Enhanced Releases

1.1.0-4.24.0

• Selective Property Serialization - Enhanced mongo and mongoFile changeTypes with selective property serialization
• Prevents unnecessary properties from being serialized in changeset YAML files
• Improves changeset file cleanliness and reduces file size
• Based on Liquibase 4.24.0
• JAR Download: Maven Repository

1.0.0-4.24.0

• Mongo Native Executor Support - Initial release with mongo native executor capabilities
• Added new mongo changeType for inline JavaScript/MongoDB shell commands via mongosh
• Added new mongoFile changeType for executing MongoDB shell commands from external files
• Enhanced MongoDB operation execution with improved performance
• Based on Liquibase 4.24.0
• JAR Download: Maven Repository

Upstream Liquibase MongoDB Extension Releases

4.24.0

• Support for Liquibase 4.24.0

4.21.0

• Support for Liquibase 4.20.0

4.4.0

• Updated code to work with core 4.4.0 PR #147
• Bump mockito-core from 3.10.0 to 3.11.0 PR#151
• Bump mockito-junit-jupiter from 3.10.0 to 3.11.0 PR#152
• Bump snakeyaml from 1.28 to 1.29 PR#153
• Bump assertj-core from 3.19.0 to 3.20.0PR#157

4.3.1.1

• Upgrade Mongo Java Driver from 4.2.1 to 4.2.2
• Fixed Issue-111: Is adminCommand really connected to the admin DB?

4.3.1

• Support for Liquibase 4.3.1
• This an important release build with 4.2.0 mongo-driver-sync and is compatible with mongo-driver 3.x.x if provided
• Fixed Issue-90: Convert all statements to runCommand so it is compatible with majority JavaDriver versions
• Fixed Issue-80: Move from deprecated mongo-java-driver
• Fixed Issue-71: Support mongodb-driver-sync v4.x

4.2.2.1

• Fixed Issue-64:Support for DNS Seed List Connection Format or Atlas Cluster
• Fixed Issue-69: Does it support preconditions
• Fixed Issue-70: Is there a way to tag the current database state?
• Added DocumentExistsPrecondition, ExpectedDocumentCountPrecondition
• Fixed Issue-74: createIndex with TTL (expireAfterSeconds) is ignored and normal index created
• Fixed Issue-79: CreateCollection silently drops supported options

4.2.2

• Support for Liquibase 4.2.2

4.2.1

• Support for Liquibase 4.2.1

4.1.1.2

• Fixed Rollback doesn't work with liquibase-mongodb-4.0.0.2 extension
• Added dropCollection and dropIndex Changes
• Added NoSql JSON Parser which can pass raw JSON for a property like this:

{
    "options" : 
        {
            "$rawJson" : { ... }
        }
}

For the command line is required to copy to [liquibase]/lib libraries : jackson-annotations-2.11.3.jar, jackson-core-2.11.3.jar, jackson-databind-2.11.3.jar

• New properties added

# If disabled can be used on API which do not support validators (Azure Cosmos DB with Mongo API, Amazon DocumentDB)
liquibase.mongodb.supportsValidator=true
# If enabled will adjust indexes and validators for Liquibase tracking tables LOCK and CHANGELOG. Can be disabled if sure Liquibase not updated.
liquibase.mongodb.adjustTrackingTablesOnStartup=true

• Overridden Liquibase table names removed. Now will be used the default ones in Liquibase. If previous releases used then table names should be explicitly passed as parameters. Currently, by default as Liquibase default :DATABASECHANGELOGLOCK, DATABASECHANGELOG Previous releases used by default : databaseChangeLogLock, databaseChangeLog

4.1.1

• Support for Liquibase 4.1.1

4.1.0

• Support for Liquibase 4.1.0

4.0.0

• Works with Liquibase v4.0.0

3.10.0

• Support for Liquibase 3.10

3.9.0

• First release

Implemented Changes:

A couple of Changes were implemented until identified that majority of the operations can be achieved using db.runCommand() and db.adminCommand()

• createCollection - Creates a collection with validator create
• dropCollection - Removes a collection or view from the database drop
• createIndex - Creates an index for a collection createIndexes
• dropIndex - Drops index for a collection by keys dropIndexes
• insertMany - Inserts multiple documents into a collection insert
• insertOne - Inserts a Single Document into a collection insert
• runCommand - Provides a helper to run specified database commands. This is the preferred method to issue database commands, as it provides a consistent interface between the shell and drivers
• adminCommand - Provides a helper to run specified database commands against the admin database
• mongo - Executes inline JavaScript/MongoDB shell commands via mongosh (Harness Enhancement)
• mongoFile - Executes JavaScript/MongoDB shell commands from external files via mongosh (Harness Enhancement)

Connection String Formats

Standard Connection String Format

mongodb://[username:password@]host1[:port1][,...hostN[:portN]][/[defaultauthdb][?options]] mongodb://mongodb1.example.com:27317,mongodb2.example.com:27017/?replicaSet=mySet&authSource=authDB

DNS Seed List Connection Format

mongodb+srv://[username:password@]host[/[database][?options]] mongodb+srv://server.example.com/ mongodb+srv://:@cluster0.example.com/testdb?authSource=$external&authMechanism=MONGODB-AWS

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.

Prerequisites

• Dependencies that have to be available in classpath if run via Liquibase CLI

mongodb-driver-sync:4.2.0
snakeyaml:1.27
jackson-annotations:2.11.3
jackson-core:2.11.3
jackson-databind:2.11.3

Installing

You can either download a released JAR from the Harness Maven repository or build the project from source.

1. Download Prebuilt JAR (Recommended)

Browse available versions in the Harness Maven repository from Registry. You can download the latest version using curl, for example:

curl -L \                                                                                                                                  
  "https://us-maven.pkg.dev/gar-prod-setup/harness-maven-public/io/harness/liquibase-mongodb-dbops-extension/1.1.0-4.24.0/liquibase-mongodb-dbops-extension-1.1.0-4.24.0.jar" \
  -o liquibase-mongodb-dbops-extension-1.1.0-4.24.0.jar

2. Build from Source

a. Clone the repository

git clone https://github.com/harness-community/liquibase-mongodb-extension
cd liquibase-mongodb-extension

b. Build the project : mvn clean install c. The generated JAR will be available under: target/liquibase-mongodb-dbops-extension-<version>.jar

3. Move JAR file to Liquibase lib Directory

Once the JAR file is available, we need place the JAR file into your Liquibase lib directory Based on Liquibase Installation method, the lib directory will be as follows:

# if liquibase is installed via liquibase website
`<liquibase-home>/internals/lib`

# if liquibase is installed via brew
`/opt/homebrew/opt/liquibase/libexec/lib`

This makes the extension available to Liquibase CLI.

---

• Run tests

Running tests

Adjust connection string

Connection url can be adjusted here: url Connection String Format Run Integration tests by enabling run-its profile

Run integration tests

mvn clean install -Prun-its

Run integration test driver backward compatibility

1. Produce test containing JAR:

mvn clean install -Ptest-jar

2. Go to test-project:

cd test-project

3. Run backward compatibility test with provided 3x driver:

mvn clean install -Prun-its,mongo-3x

Quick Start Examples

Quick start Application for NoSql liquibase extensions

Harness Enhancements

This Harness fork includes several enhancements over the Liquibase MongoDB extension:

Mongo Native Executor

• Enhanced native MongoDB operation execution
• Better integration with MongoDB-specific features
• Support for MongoDB-specific commands and operations

Additional Features

• Enhanced error handling and logging
• Improved compatibility with various MongoDB deployment scenarios
• Additional operational improvements for enterprise use cases

For detailed information about Harness-specific features, refer to the Harness Database DevOps MongoDB Documentation.

Integration

Add dependency:

For Harness Enhanced Version:

<dependency>
    <groupId>io.harness</groupId>
    <artifactId>liquibase-mongodb-dbops-extension</artifactId>
    <version>${harness-liquibase-mongodb.version}</version>
</dependency>

Original Liquibase Version:

<dependency>
    <groupId>org.liquibase.ext</groupId>
    <artifactId>liquibase-mongodb</artifactId>
    <version>${liquibase-mongodb.version}</version>
</dependency>

Java call:

public class Application {
    public static void main(String[] args) {
        MongoLiquibaseDatabase database = (MongoLiquibaseDatabase) DatabaseFactory.getInstance().openDatabase(url, null, null, null, null);
        Liquibase liquibase = new Liquibase("liquibase/ext/changelog.generic.test.xml", new ClassLoaderResourceAccessor(), database);
        liquibase.update("");
    }
}

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

License

This project is licensed under the Apache License Version 2.0 - see the LICENSE.md file for details