Versioning Data Structures in BDP Enterprise

How do I version?

Breaking and non-breaking changes

In Data Structures UI at the point of publishing a schema you'll be asked to select which version you'd like to create. There are two options:

• Non-breaking - a non-breaking change is backward compatible with historical data and increments the patch number i.e. 1-0-0 -> 1-0-1.
• Breaking - a breaking change is not backwards compatible with historical data and increments the model number i.e. 1-0-0 -> 2-0-0.

Should I choose breaking or non-breaking?

Different data warehouses handle schema evolution slightly differently. Use the table below as a guide to how to handle versioning in Data Structures for your warehouse.

	Redshift	Snowflake, BigQuery & Databricks
Add / remove / rename an optional field	Non-breaking	Non-breaking
Add / remove / rename a required field	Breaking	Breaking
Change a field from optional to required	Breaking	Breaking
Change a field from required to optional	Breaking	Non-breaking
Change the type of an existing field	Breaking	Breaking
Change the size of an existing field	Non-breaking	Non-breaking

caution

In Redshift and Databricks, changing size may also mean type change; e.g. changing the maximum integer from 30000 to 100000. See our documentation on how schemas translate to database types.

Overwriting schemas

Wherever possible we would advise always versioning the schema when making a change. However in cases where this isn't possible, Snowplow does allow you to overwrite a schema on your development environment, that is making a change and keeping the version the same.

Overwriting in your Production environment is forbidden due to the technology that auto-adjusts your tables, so when you promote an overwritten version to the Production environment you are required to increase the version as Breaking or Non-Breaking.

Incrementing the middle digit

For particular workflows you may want to make use of the middle digital as part of your versioning strategy. For simplicity, the UI allows only breaking or non-breaking changes.

Should you wish to use the middle versioning digit this is possible via the Data Structures API.