# Database sync and development workflow

**URL:** <https://forums.concretecms.org/t/database-sync-and-development-workflow/3333>\
**Category:** Developing with Concrete\
**Created:** [June 20, 2022, 11:11pm UTC](https://forums.concretecms.org/t/database-sync-and-development-workflow/3333 "2022-06-20T23:11:09Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![dustykeychain](https://forums.concretecms.org/user_avatar/forums.concretecms.org/dustykeychain/32/2729_2.png) [@dustykeychain](https://forums.concretecms.org/u/dustykeychain)\
**Post date:** [June 20, 2022, 11:11pm UTC](https://forums.concretecms.org/t/database-sync-and-development-workflow/3333/1 "2022-06-20T23:11:09Z")

</div>

Is it possible to sync data between staging environment and production environment databases for all website content (so that staff don’t have to double their workload and build pages twice) without losing submitted information, such as form submissions and unregistered Conversations block posts, from site visitors?

The reason I ask is that the current project I am working on is a Concrete site where all content and pages are built on a staging environment, and then rebuilt on the live environment once they’ve been approved. I know that non-development work, like building pages, can be done on the live site and only published when it’s ready, reducing the duplication in work, but our management team is adamant that everything is done on a separate staging environment first before migrating it to production.

Is this at all possible, or is our current workflow the only option if pages must not be built and published directly from the live environment? The ultimate goal is that no one directly touches the live environment, aside from site visitors. If user-submitted data didn’t need to persist, this wouldn’t be a problem and we’d just clone our staging to live whenever changes are made.

Any insight on this would be appreciated. We’re open to using containers to streamline the process as well if that would help. We aren’t intimately familiar with the Concrete database schema and what we need to worry about when it comes to not overwriting user-submitted data on the live site (this absolutely cannot happen) when migrating changes from staging.

---

<div class="post-metadata">

**Author:** ![katz515](https://forums.concretecms.org/user_avatar/forums.concretecms.org/katz515/32/4325_2.png) [@katz515](https://forums.concretecms.org/u/katz515)\
**Post date:** [June 21, 2022, 2:37am UTC](https://forums.concretecms.org/t/database-sync-and-development-workflow/3333/2 "2022-06-21T02:37:40Z")

</div>

I don’t know if your staging lives inside of the same server or separate server.

If it’s the same server, I’ve made the following shell script and job

[https://github.com/concrete5cojp/concrete5-backup-shell/blob/master/concrete5-copy.sh](https://github.com/concrete5cojp/concrete5-backup-shell/blob/master/concrete5-copy.sh)  
[https://github.com/concrete5cojp/concrete5-backup-shell/tree/master/application/jobs](https://github.com/concrete5cojp/concrete5-backup-shell/tree/master/application/jobs)

The copy shell copies all database and files from production to staging area, and find and replace the config of the database.php file to staging DB info.

Then, you can install either job to modify (anonymize) or delete existing users.  
You can modify the job to filter which users to modify or delete.

You NEVER want to copy production user data onto staging especially if you enable public registrations.

Anyway, if it’s different server, we have custom [deployer](https://deployer.org/) set-up. I didn’t make that one but my colleague did.  
It does more than copying the data, but deploying new code, backup, upgrade, and copying the data from production to staging. It’s very complicated to explain. But i just say we have the solution.

---

<div class="post-metadata">

**Author:** ![JohntheFish](https://forums.concretecms.org/user_avatar/forums.concretecms.org/johnthefish/32/3767_2.png) [@JohntheFish](https://forums.concretecms.org/u/JohntheFish)\
**Post date:** [June 21, 2022, 7:50am UTC](https://forums.concretecms.org/t/database-sync-and-development-workflow/3333/3 "2022-06-21T07:50:55Z")

</div>

I suspect you are looking for something like [Mainio Sync - Concrete CMS](https://marketplace.concretecms.com/marketplace/addons/mainio-sync)

Unfortunately it is not compatible with recent core versions and is discontinued. However, if you are prepared to work on the code you may be able to get it updated.

---

<div class="post-metadata">

**Author:** ![pvernaglia](https://forums.concretecms.org/user_avatar/forums.concretecms.org/pvernaglia/32/3579_2.png) [@pvernaglia](https://forums.concretecms.org/u/pvernaglia)\
**Post date:** [June 23, 2022, 12:22pm UTC](https://forums.concretecms.org/t/database-sync-and-development-workflow/3333/4 "2022-06-23T12:22:15Z")

</div>

For new pages you might look at the migration tool, if you don’t have a lot of custom blocks it might save you some time.

---

<div class="post-metadata">

**Author:** ![dustykeychain](https://forums.concretecms.org/user_avatar/forums.concretecms.org/dustykeychain/32/2729_2.png) [@dustykeychain](https://forums.concretecms.org/u/dustykeychain)\
**Post date:** [June 23, 2022, 8:13pm UTC](https://forums.concretecms.org/t/database-sync-and-development-workflow/3333/5 "2022-06-23T20:13:54Z")

</div>

This looks to offer some of the functionality I am looking for, definitely. I mostly want to bring page data forward from one environment into another via the database, with everything else other than package installations and version updates being handled with source control, wherever possible. Do you see any issues there?

Also, this add-on is currently disabled. Is it still possible to purchase a license? We are currently still using the v8.x core, so it doesn’t need to be v9 compatible for our use case.

---

<div class="post-metadata">

**Author:** ![JohntheFish](https://forums.concretecms.org/user_avatar/forums.concretecms.org/johnthefish/32/3767_2.png) [@JohntheFish](https://forums.concretecms.org/u/JohntheFish)\
**Post date:** [June 24, 2022, 8:12am UTC](https://forums.concretecms.org/t/database-sync-and-development-workflow/3333/6 "2022-06-24T08:12:58Z")

</div>

AFAIK, the Mainio Sync tool was discontinued at v7 because changes to the core at v8 prevented it from functioning and the work involved in updating it was disproportionate to the value of the project. I believe it was originally developed from the migration tool @pvernaglia refers to.

You can’t purchase it from the marketplace. You would need to contact Antti Hukkanen [Contact / Mainio Tech](https://www.mainiotech.fi/en/contact/) to get a copy, then dig into the code to get it working for v8 or v9 sites.

Back at v7 it was a good tool and worked reliably within its scope of copying between sites with identical applications/packages.
