Hi everyone,
First thing, take a quick glance at this site:
https://d39v4abs5ngx4v.cloudfront.net
There’s your pretty default V9 install, nothing unusual, the search works, etc.
There might be a few odd pages that don’t work, but overall it should be fairly fast and seem like a normal install.
I can log into this and edit it fine, add pages, edit blocks, visit the dashboard, etc.
But the interesting this is that this test copy is not running on a server as such, it’s running in a ‘serverless’ configuration.
If you’re not familiar with what ‘serverless’ is/means, it’s where instead of you spinning up a server and running something like Apache/Nginx, and having that server constantly running waiting for requests, you instead deploy your code to a service that effectively runs your code only when a request comes in.
In this particular case this site is running on Amazon Web Services, using their ‘Lambda’ service to actually run the PHP (there’s more to it than that, but that’s the main thing in play here).
The benefits then are:
- you don’t actually have a server to monitor and maintain. It won’t go down due to lack of disk space, crashed processes, filesystem corruptions, etc
- it can handle a large number of requests simultaneously, it scales automatically (and you can also have scaling databases too)
- you have effectively unlimited storage space
- security is arguably higher, as there’s no server to exploit
- costs can be quite low, as you only pay for the time your code is running
Now all these points have caveats, and there are quite a number of potential drawbacks, but that’s the overview at least.
For a PHP framework like Laravel, it works great serverless, and we’ve had a lot of success running some mission-critical apps this way. I’ve actually moved some from being on normal servers over to serverless and haven’t looked back.
But I’ve often wondered if Concrete could be run this way, especially when Concrete and Laravel share a lot of the same DNA design and code wise. Hence setting up the little experiment.
So how does this work?
There’s a framework just called ‘Serverless’, which is where you describe what you’d like to run in a yaml script, resources like S3 buckets, and even databases, and it then automates the process of setting this up somewhere like AWS.
Then there’s a project called Bref, which provides a PHP runtime for Lambda. This is effectively a plugin for Serverless, and with both together allows you to run PHP scripts serverless. (if you are interested in serverless PHP, start here, and read the documentation thoroughly)
And finally I’ve used a related project called Lift, which further makes automating some of the common patterns of setting up a website easier. It was after realising what Lift did that I realised it might be possible to actually run Concrete serverless.
The final script is actually only about 50 lines, and most of that is copy and paste config that is quite generic and easy to read.
What it does when it deploys:
- when the deployment is triggered it packages up all of Concrete’s code into a zip, pushes it up to Amazon’s Lambda and API gateway, configuring Bref to actually run the PHP code.
- Separately collects static files, like in /concrete/js or /concrete/images, and sends them up to an S3 bucket
- set up Amazon’s Cloudfront (which is like a CDN), to serve the static files from S3, but send other web requests to Lambda to be run (so actually like a normal webserver)
There’s also a database set up for this, but that’s not serverless, just created separately.
I’ve not added a custom domain here, but that’s not hard to do with Cloudfront.
Emailing should be quite easy to set up, as it’s just a case of using Amazon’s emailing system.
However, there’s quite a lot of issues to resolve:
- Think of Lambdas like little instant virtual machines that spin up, run your code, then disappear. They don’t have a persistent file system, meaning you can’t write anything to them at all. The only place you can write is to /tmp, just for things like processing files, but that disappears at the end of the request. So this means that I’ve had to configure Concrete’s cache to write to /tmp instead of in the /application/files directory (not too tricky)
- Because sessions write to the filesystem by default, I’ve I also had to change sessions to be database driven (very easy, fortunately)
- Concrete’s File Manager also expects to write files and thumbnails to /application/files, so you have to install an S3 driver to read and write to an S3 bucket instead. Ordinarily this shouldn’t be a problem, but my S3 add-on (which admittedly was written a while ago) seems to still want to write to the filesystem for some reason, so I haven’t got this working yet. In theory it should work, but may take some fiddling.
So then the hard stuff (that might just not be solvable)
- Any configuration that writes to /application/config simply doesn’t work, as it can’t write to those files (and if it did, it would disappear on the next request). Put another way, code is effectively frozen when you deploy an application. That’s great when you’ve got something like a Laravel application, where you don’t expect the code to change, but when config is stored in code, it then makes it impossible to be able to change settings without doing that somewhere else and triggering the serverless deployment again. The solution would be to store all overriding config in the database, but I don’t think that’s currently an option.
- Concrete has a way to have environment specific config files, which is something I might take advantage of here (but won’t solve the problem above)
- Not being able to write to the filesystem also means you can’t install new add-ons easily - you have to push them up via the serverless deployment, and then they might install ok, but many will do things like write their own config or trigger the creation of Doctrine files in /config/doctrine, so you quickly hit issues. Turning off Doctrine caching might make this easier, but it’s going to slow things down.
- New packages you install are going to have their own static resources, js, css and image files, and they need to be specified in the serverless config and pushed up.
- The sitemap.xml file can’t be written to the server either.
What I’ve actually got is a local Concrete install, one that I first set up, turned all the caching off for, set up the specific configuration and pushed up.
So to make further changes, I have to make these changes locally, push those up and hope it still works.
Now with the challenges remaining to have everything working as expected there might be no real value in continuing with this experiment. It might be Concrete just isn’t quite suited to running serverless.
Perhaps I could imagine a set up where you have a staging copy on a traditional server, where you set things up, add new add-ons, etc, and you then deploy from that to a production serverless environment, where everything is very locked down and highly scalable.
But even if I’ve sort of hit a dead end, I thought this would still worth sharing, as it might give others some ideas, or simply find interesting.
Cheers
-Ryan