Verify new block types and copy blocks between websites

[mlocati]

Hello everybody,

I’m writing here to share a new package I recently developed which may be useful: Blocks Cloner (available on GitHub - open source with MIT license).

It’s a package that allows you to “copy” blocks between different websites.

It works with concrete5 v8.5 and ConcreteCMS v9.

I think this package can be useful for:

• people who want to copy contents from a website (an old one, a staging site, …) to another one (the new one, a production site, …)
• package developers who want to check if a new block type correctly supports exporting/importing blocks in CIF format (based on XML)

Here’s how it works:

Export

Once you install the package and you edit a page, click a block: you’ll see a new “Export as XML” menu item:

If you click that item, you’ll be able to view and copy the CIF that defines the block and its data:

You can also pick the block to be exported by clicking the following icon that you’ll see in the toolbar:

If you click that icon, you’ll see a panel where you can see the whole page structure and click the block you want to be exported:

Import

Once you have copied the CIF, you can “paste” it into a specific area by clicking the new “Import Block from XML” context menu item:

When you click that item, you’ll be presented a dialog where you can paste the CIF text, and optionally choose where the new block should be inserted (before another block that exists in the area):

When you click the “Analyze” button, the code checks if there are some errors (invalid XML, unknown block types, access denied for adding that block type to that area, …).

If the checks pass, you’ll be able to actually import the new button:

In order to pick the destination area, you can also click the following new button you’ll see in the toolbar:

If you click it, a panel will appear, allowing you to select the area where you want to paste the CIF of the new block:

Remarks

1. this package is still in a rather early stage (I still have to take care of some stuff like handling missing images)
2. this package requires that blocks correctly support importing and exporting data in CIF format: don’t blame me if a block developed by someone else doesn’t work as expected

[mlocati]

PS for the PRB reviewers (@JohntheFish, @mnakalay, @enlil, …) you could suggest package submitters to use Blocks Cloner when they develop new block types to see if everything works as expected (and ideally you can use Blocks Cloner to check packages submitted to the marketplace as well).

[mhawke]

What a wonderful addition. Thank you.

[jb1]

That’s an incredible add-on! Well done! A really clever way to handle granular cross-site content import. I can think of quite a few scenarios where that will be useful.
I still have an old (but big) 5.6.x site I haven’t finished migrating to the latest version because each custom block has to be re-built from the ground up (and tested). This tool will make the process much easier.

[mlocati]

@JohntheFish what do you think about adding this package to the Concrete marketplace?

[JohntheFish]

I am very much in favour of this as a concept. I have implemented a settings export/import/save etc… on my own advanced blocks, but nothing generic like this.

I have been following your GitHub updates to CIF etc and was wondering if such a tool could become feasible, so am delighted to see you working on this. The ability to copy from, for example, a stage site to a live site, is something that many customers request, especially enterprise customers who want to manage a stage to live workflow for new content.

I have not had a chance to play with it yet. Risking some feature creep here:

• What happens if it refers to files or images? could it export/import a zip containing XML and other files? OR could it refuse to import if the assets were not in the destination file manager?

• What happens if there is block design? What happens if the design refers to background images?

• What happens if there is a stack (or other grouping block such as layout or container)?

• Could it be extended to work with a complete area? Then on import provide a choice to either insert or replace the destination site’s area (this is the capability my enterprise customers would really like)

• Which leads me to the corollary of that, could it be extended to summarise a diff between exported area and previous target area?

For the marketplace, I think some big customers would pay well for this if it handled most/all of the above in a robust way.

Thinking back, there was a previous Sync addon that could push pages between 2 identically configured sites. It was great in concept, but from my point of view was hobbled by the sites needing to be identically configured. It couldn’t, for example,work with various developer packages installed on a stage site but not on the live site.

[mhawke]

It’s funny (actually kind of pathetic) that I extended the code to include the block’s files in a nice neat ZIP file that I could download and install on the target site but then realized I could just FTP the damn block. I’m getting too old for this but it was fun trying to extend this. Keeps my brain semi-working as I age.

[mlocati]

@JohntheFish

About files/links/images: when you export something, the export dialog allows you to download the used files.
When you then import the XML, we look for referenced files/images/links/…
If some file can’t be found you can upload the ones you previously downloaded from the export dialog.

About block design: it’s exported and imported correctly (yep, including background images). But in order to view the block design styles at the moment you have to refresh the page: I’m going to fix this asap.

About stacks: it’s in my to do list.

About exporting all blocks in an area, as well as area design: it’s in my to do list.

About cleaning all blocks in an area before import: it’s rather dangerous, I’m not planning to do it.

About area diffs: I’m not planning to implement such a complex feature (implementing it would require a ton of time, and I don’t have that much spare time).

[JohntheFish]

@mlocati

About area diffs: I’m not planning to implement such a complex feature (implementing it would require a ton of time, and I don’t have that much spare time).

I wasn’t thinking of a fully rendered page/area diff (the core functionality for that is flakey for non bedrock themes). More of comparing the source/destination XML and from that extracting a list of blocks highlighting which blocks are different, not the details of the update. A bit like the overview display in your JavaScript package diff.

[mlocati]

@mlocati

About block design: it’s exported and imported correctly (yep, including background images). But in order to view the block design styles at the moment you have to refresh the page: I’m going to fix this asap.

Done: block design is imported and displayed correctly.

@mlocati

About stacks: it’s in my to do list.

Done (but requires Fix core_stack_display export/import - Take 2 by mlocati · Pull Request #12508 · concretecms/concretecms · GitHub for an improved support).

Everything should work both in normal areas, global areas, and in the “Edit Stack” dashboard page.

Block aliases are exported as non-aliased blocks (for v8 we need [v8] Backport core scrapbook display export by mlocati · Pull Request #12506 · concretecms/concretecms · GitHub).

The next thing I’ll do is implement export/import of area contents and area styles.

[mlocati]

@mlocati

About exporting all blocks in an area, as well as area design: it’s in my to do list.

Done: I’ve published version 1.0.0 - see Releases · concrete5-community/blocks_cloner · GitHub

[jb1]

Amazing! I’ve just tested it out on the same site - exporting an area to XML and then importing that XML and it’s a seamless experience. Intuitive and elegant. Well done!

[mhawke]

On one Content block that was pulled from the Clipboard but had not yet saved as a new block (so I’m getting the warning about the "block was copied from another location. Editing it will create a new), I’m getting a “Call to a member function isBlockAliasedFromMasterCollection() on null”

If I edit that block and save it as a new block, the export works well.

[mlocati]

I’ll do some testing. Thanks for reporting the issue, Mike

[mlocati]

@mhawke which ConcreteCMS version are you using?

[mhawke]

It’s on Version 9.3.9

[mlocati]

@mhawke

I’m getting a “Call to a member function isBlockAliasedFromMasterCollection() on null”

I’ve been able to replicate the issue, provided that the orginal block that was copied is deleted.
If this is the case, it’s a problem of Concrete fixed through Fix exporting aliases of deleted blocks by mlocati · Pull Request #12519 · concretecms/concretecms · GitHub