Search Index for Express Objects

Hi there, I’ve developed a site using v9 and express objects extensively. It’s a historical society. In all three cases, I use the Express Object List to give the visitor options to click any record to read more. The URLs are all fine but my issue is that the detail pages are not indexed in the site search engine.

An example URL that I would like to appear in the site search :
https://www.nanaimoarchives.ca/transcripts-and-recordings/historical-audio-recordings/view-recording/view_express_entity/12

Any way to make that happen? Yes, I realize that I can manually create all the pages and install the express detail block and link them up … but that’s a ton of work. There are hundreds of records. Any ideas?

1 Like

Hi @fridayphoto,

This is a good point, probably something Express should support in the future. I definitely recommend making an issue for it as a Feature Request here because I think a lot would benefit from it and I’m sure it would get some traction.

In the meantime, I think your best bet would be a custom search that searched the appropriate express object and then figured out its frontend detail page and linked to it.

That is the kind of problem this issue is aimed at resolving. Please add your thoughts.

So I’ve gone through all the trouble of creating hundreds of pages and configuring a detail block on each page and the content still doesn’t show up in a site search. I’ve rebuilt the indexes, makes no difference. It appears that ‘Express Detail’ content is not indexed. Does the search index only work on ‘Content’ blocks? The attributes with the important text have the check mark for ‘Content included in search index.’ but I guess this is only a search index for this object only, nothing to do with the sitewide keyword search. Ugh. I’m failing at finding a solution so you can search the express object data from the sitewide search for website visitors. Any ideas?

I’ve read about using “getSearchableContent” to index content from a given block. I tried adding the following function to the /concrete/blocks/express_entry_detail/controller.php but it didn’t seem to do anything. Reindexed then tried the search again. Anyone know how to actually make this work?

public function getSearchableContent() {
        $express = \Core::make('express');
        $controller = $express->getEntityController($entity);
        $factory = new ContextFactory($controller);
        $context = $factory->getContext(new FrontendViewContext());
        $renderer = new Renderer(
            $context,
            $form
        );

       return $this->set('renderer', $renderer);
    }

Your getSearchableContent() method needs to return actual text, not a $renderer for the text.

At a guess, there will be some method on the $renderer that spits out the text, or maybe a __toString() magic method that does that.

Maybe I’m misunderstanding, but it sounds like you are trying to do something like this: brand_central/src/AssetLibrary/API/V1/Assets.php at f84a29308f2c8d12a72a035cdcd1c29345812575 · concretecms/brand_central · GitHub

Thanks for this. There’s quite a bit to digest in what you’ve sent me. I’ll have a look at it and get back to you.

1 Like

Thank you JohntheFish, cubit_07sorrel & DL0rnie - I lack the genius to instantly implement the ‘AssetLibrary’ code that was provided but this seems like it should be very helpful and I’ll confer with my developer. To reiterate, we just want the text data that displays on manually created pages with Express Detail blocks to be searchable from the top right sitewide search. I’ve already manually created all the pages and configured all the detail display blocks. I think the answer lies within the express_entry_detail/controller.php but this is a bit outside my expertise. Any implementation tips by ConcreteCMS geniuses would be hugely appreciated.

I wish this hadn’t taken this long to get to you, but hopefully this is better late than never (especially since after our chat you might be heading away from the Express Detail approach anyway.) Perhaps this will help someone else - I will also be opening this as a pull request for 9.2.2.

I think this method should get you what you need:

public function getSearchableContent()
{
    // The entity manager doesn't appear to be available in the CLI version of task running? At least not
    // at the block level when retrieving searchable content? So let's manually create it.
    if (!isset($this->entityManager)) {
        $this->entityManager = $this->app->make(EntityManager::class);
    }
    // Let's run the view() method so we can populate the renderer object and the entry that we're supposed
    // to render.
    $this->view();

    // Now, if we do indeed have the renderer and entry defined within the block controller, let's retrieve them.
    if (($renderer = $this->get('renderer')) && ($entry = $this->get('entry'))) {

        // Since we have the entry and renderer, let's actually render the thing. This will result in printing
        // out actual HTML, so let's output-buffer it.
        ob_start();
        $renderer->render($entry);
        $content = ob_get_contents();
        ob_end_clean();

        // Finally, let's do some quirk and dirty removal of HTML and the massive newlines that
        // such an operation will create. NOTE: This is not meant to be perfect or meant to look
        // particularly beautiful if rendered - it is meant to populate the search index with an approximation
        // of the content that's actually being shown on the page.
        $content = str_replace(["\r", "\n"], " ", strip_tags($content));
        return $content;
    }
    return '';
}

When this method is added to the Express Detail block’s controller, it takes a page like this:

(which is built using an Express Detail block, pulling from some fields stored against an employee record in the database)

and stores the data like this in the database:

And then, when you use the built-in search, keywords from this now properly show up in the search results:

Pull request added here Add getSearchableContent method to Express Detail block by aembler · Pull Request #11555 · concretecms/concretecms · GitHub

Thank you so much for this Andrew!

I’ve installed this code. First I tried an override in /application/blocks/express_entry_detail/controller.php and when that didn’t seem to do anything I thought I might be doing it wrong and just put the code direct in the /updates/…/controller.php for v9.2.1

Both times I’d install the code, flush the server cache, flush the ConcreteCMS cache, run the reindex task. Both times, none of the express driven pages showed up in the PageSearchIndex table. The cIDs for all the express-driven pages don’t appear in the index at all.

I’ve been working with cID 723, and even tried installing a text block on the page. Still no dice. No appearance in PageSearchIndex.

If you’re going to override the controller you’ll have to disable the overrides the cache in the dashboard and then change the namespace of your file to

Application\Block\ExpressEntryDetail

Honestly - I know it’s bad practice but if you want to just quickly test it out I’d just add the method directly to the controller found at concrete/blocks/express_entry_detail/controller.php

Thanks for the follow up Andrew. I’ve tried adding the new code in the following:
/application/blocks/express_entry_detail/controller.php
/updates/concrete-cms-9.2.1_remote_updater/concrete/blocks/express_entry_detail/controller.php
/concrete/blocks/express_entry_detail/controller.php

No change. When I rebuild index, none of the pages under ‘Finding Aids’ appear. (cID’s 723+)

Do you think this could be related to how I create the pages? They were all created with migration tool.

I even created a text block with content on cID 723, and it does not appear in search index.

I double-checked and none of those pages have ‘Exclude from Search Index’ attribute.

Hi Andrew,

This looks great for when we are using the Express Detail Block with an express attribute on a page. I’m working on something right now where I am linking from the Express List Block to a single page with the Detail Block and am looking for a way to have those linked pages show up in search.

I was thinking maybe the list block could add to the search index if there was a way to pass the the express ID that appends the url. I don’t think there is a way to do that now?

Thanks

I see in the github update that Andrew made there’s a new line near the top of the controller.php

The line “use Doctrine\ORM\EntityManager;”

I tried just adding this line but it still doesn’t resolve the issue.

It works! I was too focused on errors and didn’t check the SearchIndex table after adding the line

“use Doctrine\ORM\EntityManager;”

I just checked the SearchIndex and it’s populated with all of the express entry data! Wahoo! Thank you Andrew!

1 Like

Discussion about this on Find Out Friday - Youtube

1 Like

I am trying to get Searching working for my Express Object using the above idea. I cannot seem to get it to work. Can someone help me by providing a step by step procedure to get this to work, I am using 9.2.2 and the Express Detail block. Thanks!

Here are errors I see in the log after adding an express detail block to a page:

19 	application 	Info 	Rejected message Concrete\Core\Page\Command\QueuedReindexPageCommand will be sent to the failure transport Symfony\Component\Messenger\Bridge\Doctrine\Transport\DoctrineSender. 	12/4/23, 9:39 AM 	admin 	

18 	messenger 	Alert 	Messenger Worker Message Failed: /var/www/html/pac/concrete/vendor/symfony/messenger/Middleware/HandleMessageMiddleware.php:130 Handling "Concrete\Core\Page\Command\QueuedReindexPageCommand" failed: Could not convert database value to 'object' as an error was triggered by the unserialization: 'Failed to save key "Concrete__Core__Entity__Attribute__Set__CLASSMETADATA__" of type Doctrine\ORM\Mapping\ClassMetadata.'
12/4/23, 9:39 AM 	admin 	

Thanks,
Joaquin