Iterating through Express Entries for single Object, stuffing various results into arrays, then drawing... keeping fidelity?

[BloodyIron]

I’m trying to build a rocket ship, so to say, and I am not exactly an Engineer. This is a metaphor, somewhat, to explain that I have had to teach myself a lot of stuff with Concrete CMS, to achieve what I want. And to preface that there’s plenty I do not know.

Now, I will try to describe what I’m trying to do here, but I also want you, the reader, to keep in mind that I may be missing something obvious to you, just because I “am not an Engineer”, so to say.

I am trying to build a custom template for an Express List Block, but the issue is more about trying to write code that “scales well”, but not to the ABSOLUTE MAXXXXX. Just… trying to do some well-thought-out advanced planning.

I have an Express Object “Event Schedules”, and the Entries in it have upwards of 8 different associations to consider (may even be more in the future?).

Each Entry is generally expected to only have one association relevant, and only select one associated Express Entry that is in another Express Object.

The Express List Block customisation I’m trying to do is one big “Schedules” block, where it shows all the different schedules, grouped together, based on what the association is. But also checking if the schedule is relevant to “this event” (and I think I already know how to make that determination).

The problem that I am grappling with is that it is HIGHLY PROBABLE that as the loop iterates through the Express Entries for the Schedules Express Object, that the order will be chaotic, and not in any way already grouped based on “which” Association. This gives me a problem about grouping the results together on the page. From what I can tell this is because PHP generates HTML (when it does) in-order, and I’m not really seeing a way to add HTML “higher” on the page, once that part of the page is already “generated”.

I want to run the first loop only once, as a goal, for scaling and performance issues. As over time the Express Entry count for this Object is expected to grow more and more over time, and I want to minimise that impacting performance on page generation. So instead of running the loop that churns through the Express Entries for EACH of the associations, I would prefer to do it once, and process the whole list… once… then do useful things with it.

Now…

My current working “theory” is to iterate through the Express Entries once, and when I get a “hit” for each of the Associations, I stuff that “hit” into an Array for that Association, and I guess I have one new array for each Association. And THEN after that loop I run another loop for each of those arrays to then pull that data out and render the sections, grouped together, for each “Association” so to say. So that the visual presentation makes sense to humans.

The part that I am unsure of, because “I am not an Engineer” is that… when I stuff the “Entry” result into a new Array… does the full “fidelity” become retained? In that, when I refer to that value in the Array, are all the Express-y-things retained? Can I ->getAssociatedThinggy->getAssociatedThinggyAttribute etc… on each of the things in the new Arrays? Or is that fidelity lost?

I’m open to different methods here, but I also REALLY need to know if this “kills the frog” so to say, by “copying” the result into a new array. Or… maybe it… “just works”? (~.^~MaGiC~.^~)

Thanks for your time and whatever insight can be shared on this matter!

[BloodyIron]

I just tried to dig up a way to do this with php, so that I keep the full fidelity of each $item in a “foreach(result->getItems() as $item)” loop, but also only copy certain ones.

So far I’m only really finding examples that copies the whole array, and well… that’s not what I want.

Can anyone jump in here please? I have a hunch there’s a “basic” solution to this, but I’m not yet seeing it.

[Parasek]

You should filter data by association on list level, like:

use Concrete\Core\Entity\Express\Entry;
use Concrete\Core\Express\EntryList;
use Concrete\Core\Support\Facade\Express;

$entity = Express::getObjectByHandle('tournament');

// Argument here will be singular or plural, depending on association type
$association = $entity->getAssociation('tournament_registrations');
// You need to get Entry object (or id) from somewhere,
// for example from another list or hardcode it directly
$associatedEntry = Express::getEntry(8);

$list = new EntryList($entity);
$list->filterByAssociatedEntry($association, $associatedEntry);
//$list->debug(); // This is only for checking sql query
$entries = $list->getResults();
//dd($entries);
/** @var Entry[] $entries */
foreach ($entries as $entry) {

}

and make separate list for any chosen association type (instead iterating all records from database).
And preferably paginate results, to not display them at once.

[BloodyIron]

1. You are glorious, I didn’t even know something like this was possible, or that it was preferable from a performance/scalability perspective!
2. What about this method is preferable, from a performance/scaling perspective, over say, running loops through the Express Entry list for each association? I can speculate to a limited degree, but probably not sufficiently, on the difference between what you propose, and alternative methods.
3. “$languageEntry = Express::getEntry(10); // ID of language entry (for example from clicked field)” I don’t understand what this “is”. To add some clarity, this list is expected to generally be non-interactive. In that it is a visualisation of entries in nice ways and such. So I don’t see how this “works”. … “Express:getEntry(10)”… what exactly is this… referring to? (like, is this expected to be the Express Object that the Express List Block is configured to use? So that is… implied reference as to which Express Object this references? Either way, I don’t understand what this part is, how it works, etc.
4. How well does this method handle NULL association values? Will I need to bake-in if statements for “!== NULL”?
5. This $list, does it become an array of references to the original data? Or is it copies of all the data, and becomes a separate “object” from the original Express Object/Entry? (I may not be using the correct terminology here, this references the Fidelity aspect)
6. When doing the “$list->filterByAssociatedEntry(…” does that filter OUT anything that doesn’t result in “$association == $languageEntry”? Or what is the expected behaviour for this step?

Thanks yet again!

Oh and for the form stuff you’re helping me with in another thread, I still am going to try with your method, but I temporarily shifted gears to this for now (but this task ended up being more complicated than I originally thought, oof).

[Parasek]

1. I have updated example in previous post.

2. Methods added/chained to $list just add some SQL code to query.
   You can actually check it by using:

   $list->debug();
   

   to see how query is changing when you added different filter/sorting methods etc.
   This is also applicable to any list, like Page List, File List etc., since they share some part of API and work very similar.

   When you fetch all records from database and loop it, you will be probably making additional query for every iteration in loop when asking for association.
   By moving that part to single sql query, you are not only lowering number of queries, but decreasing number of objects you have run through.
   Databases are designed to hande select statement efficiently (indexes and stuff like that), so with big enough number of entries it should be much more faster.

3. You should know entry id by looking at second express table.
   You can just visit /dashboard/express/entries dashboard page, find associated entry from related table an look at url in browser, like /dashboard/express/entries/view_entry/8.
   Last part is an “id” that you are looking for.
   All express entries are placed in the same db table (so id is always unique across all express objects).

   If you want too look only for specific associated entry you have to “kinda” hardcode that number.
   “Kinda”, since you can add custom/unqiue “handle” text attribute and find specific $associatedEntry by it (by making another $list).

   Or you can just make separate list to get all associated entries (from second table).
   Then you loop through it and inside foreach you would get $associatedEntry object directly.

4. Should be fine. It’s just sql query under the hood.

5. You can use dd($something) to see what kind of object/what class $variable is “using”.
   $entity just hold info about specific “express table”.
   $list is just preparing sql statement and you will only get data when you run $list->getResults() or pagination method.
   By data, I mean an array of Express Entry objects (where you can use different php methods etc.).
   Since it create object for every result item, it is good idea to paginate results (unless you really need to fetch full list), though you will probably not see a difference in smaller database.

6. Yes.

[BloodyIron]

Okay so I see the wisdom in what you propose here, namely due to the point that you make that each “foreach” loop results in a query each time (I thought it was already in-RAM), and filtering out sure does look to scale better!

That being said, it looks like there’s a couple details here where you made some assumptions that doesn’t line-up with my goal, but that’s mostly my fault for a bit of ambiguity (sorry), so I’m going to clarify on a few details. And I’m hoping to get your input again as I’m not sure the “best” way to adjust your recommendation with the “new” considerations.

So…

1. This Block Template is expected to be used on pages where a page Attribute of Express Entry type is present, and has selected “which event” so to say (event is an Express Entry in an Express Object for events). The page is expected to be the “event page” so to say. And this block is one of many blocks on the page that adjusts based on “which event” is selected.
2. For each of the “lists” ($list1, $list2, etc) my first intent was to filter out Express Entries that did not have any Entry selected via the “Target Association”. I “don’t care” which Entry is selected in this filtering process, just whether “This Association has a Target Express Entry selected” at all. This is namely for presentation purposes, as each “Target Association” is going to be visually grouped (and titled) based on that association, think of it like categories. So, for this “category” naturally any Express Entries that do not have any Entry selected for “This Association” (for “this” category) are irrelevant, and filtered out. And any Entries that do have an Entry selected for “This Association” (it is always expected, in this case, that each Association is SINGULAR in Entry Selection, and never MULTIPLE for Entries).
3. My next intent, in this method, is to then take the $list1, and drill into the Selected Entry for the “Target Association” and check if that “Selected Entry” itself is relevant to this event by ->getEntry()->getEvent() kind of drill down, and then comparing it to the “which event” Page Attribute described in #1 above. And then when I get a “hit” actually start pulling data out of “This Entry” and visually populate “this” part of the Div/Table/whatever. And then iterate to the next Entry in $list1/2/3/ whatever.

So, as a result, the part “$associatedEntry = Express::getEntry(8);” (example) does not work for my use-case, in that initially I don’t care which Entry is selected for “Target Association”, but that it does, or does not, have any Entry selected at all. And then do useful stuff with it once I’ve identified that.

So I’m not currently seeing how to adapt the method you propose considering this. What would you recommend?

Again, this is really helpful! Not only the method, but the additional explanation. I really appreciate you helping, yet again, spending your time, and giving the added detail. This is helping me in multiple regards! And hopefully other people too as they see this thread (and others) over time.

[Parasek]

Given two Express objects:

• Event
• Category
  and Event is in relationship with Category (Event can have one Category selected),
  to display list grouped by category, you want to:

1. Get list of all categories. That’s one sql query.
   Make foreach loop foreach ($categories as $category) where $category is an $associatedEntry from example.
2. Inside forach loop write another list (example list from previous post). This will result in as many sql queries as there are categories.

Btw, your approach is fine if you filter entries by given Event first, since it will be probably a small set of returned entries anyway. If you display them all anyway, then it will be similar number of queries.
But in case you do the same for larger collection of data (where pagination is required etc.), it would be not as efficient as filtering list in the first place.

[BloodyIron]

I don’t think we’re quite yet on the same wavelength, I may have mislead you with some details earlier, unsure. But let me add some even more clarity.

So, in this goal, there are 2x Express Objects primarily relevant, and about 7x other Express Objects secondarily relevant.

Primary Express Objects (each of these are an Express Object):

1. Events
2. Scheduling

Secondary Express Objects (again, each of these are an Express Object, consider each of these a different “section” of an Event, so to say):

1. AssocObj1 (for the sake of example, Tournaments)
2. AssocObj2
3. AssocObj3
4. AssocObj4
5. etc
6. etc
7. etc

The Express Object for Scheduling has some standard Attributes, date/time, title, etc. But each Express Entry is expected to have only ONE Express Entry selected from any of those 1-7 Express Objects (so one schedule entry is for a tournament, another could be for AssocObj2, or whatever). But each Express Entry for Scheduling does not itself necessarily directly associate (or select an Express Entry) for an Event.

So, as mentioned previously, on the Event Page, there is an Attribute of type Express Entry, selecting “Which Event” as in which Express Entry for the Event intended for the page.

With the filtering method you proposed, what I’m hoping is to go through the list several times to filter into $list1 $list2 … $list3 or whatever.

Each $list# will have Schedule Express Entries that have an Express Entry at all (boolean check) for AssocObj# Association. So roughly $list1 Express Entries is expected that each will have an Express Entry selected for AssocObj1, and then $list2… for AssocObj2… etc.

Then, once each of those $list#'s are populate, iterate through each, and drill into the Express Entry that’s associated, and determine if that Entry is for this Event. So getEntry()->getAssocObj1->getEvent() == $PageThisEvent

When creating the Express Entries for AssocObj1…7, those Entries will pick the relevant Event.

Now. This is what’s in my head currently, and I hope this clears up misunderstanding you may have.

How can I filter out results when creating $list# such that those in the list are conclusively known to have any Express Entry selected for AssocObj#? (filtering based on boolean check, does it have an Entry selected at all?)

In this case, all Express Entry selections through Association is expected to be a SINGULAR Entry selected, not multiple.

Does that make sense? I hope so D:

Thanks again! And I’m open to other ideas if you have any.

[Parasek]

1. What type of relationship is between “Event” and “Scheduling” objects?
2. You want to display your stuff on “Event” page (so only data related to specific Event), right?

[BloodyIron]

1. “Events” and “Scheduling” don’t have an Association in this case.
2. Yes this is intended to be on an “Event” page.

[Parasek]

I’ve started questioning if you actually properly modeled objects in first place.
Try to describe what do you want achieve without technical jargon and without giving answers in questions. Something in line:

I have list of events in database ("IEM Katowice 2023", "ESL Challenger Melbourne 2023 " etc.).
I have list of tournaments ("CSGO Finals 2023", "LOL SEASON 123" etc.).
Each event can host multiple tournaments. Tournament is one-time "thing" occurring only at one specific event.
..here describe something about Scheduling and how it fits here...

I want to have an Event page on my website, where list of XXX is being displayed.
It will look something like that:
...example how will it look on Event page...

and so on.

You don’t have to give many details, but one important thing is to describe how these “elements” are connected with each other (and how/where you want to display data from them).
This is the foundation I can work with it (and give proper answers).
Rest is just technicalities.

[BloodyIron]

Sorry I’m not trying to frustrate you, so if that’s what’s happening anyways, sorry

This is part of a bigger amount of work to build a new event system and various components, so we’re not migrating any data at all. We don’t “have” events just yet on this “new way”, just on an old website that we will not be migrating data from.

In this particular case, I am trying to create a nice first-version of presenting a big “list” of schedules for each event, grouped together based on “sections”, and “tournaments” is one section. There are other sections too, such as official broadcasts, and partner broadcasts (these are intended to be treated as separate sections), plus other sections of other names.

So on an event page, I will have a section specifically dedicated to schedules, whereby it shows all the things scheduled for that event, so it’s easy for the viewer to know what’s coming up, what’s going on now, what’s already happened, etc.

Does that clear things up?

[Parasek]

No worry, I am just asking some questions:D

Do you even need “Schedule” object? What fields do you have in that object? It sounds like it’s not necessary at all… or maybe it is needed, depending on what do you want to do with it.

[BloodyIron]

I really do see value in the Scheduling object. Even though it is “simple” in terms of how many attributes each entry has.

• Title
• Note
• Start date/time
• End date/time
• Then association stuff

This is valuable because I don’t want to limit “how many” Schedules can be associated with a single “Thing”.

This is most notable for multi-day events, where a Tournament, or something else, can have multiple schedules for different “stages” of the tournament, or for other reasons.

In-contrast, if we have the (for example) Tournament Express Object itself have Attributes for each Start/Stop, there’s some problems with that.

1. We have to keep modifying the Tournament Express Object if we want more scheduling attributes for more complex Tournaments in the future.
2. The UX when defining a Tournament becomes less than ideal, vs the “create schedule for Thing” perspective.
3. Making a single “visible pane of glass” for all the Schedules of all the things for an Event means I need many more Express List Blocks on the page, each with their own Template, etc.

So yeah, I still do genuinely believe having a Scheduling Express Object is the preferable thing to do. It centralises where such things are kept, maximises flexibility, and IMO is a preferable UX for creating entries.

[Parasek]

Event should have One-To-Many Association (target: Tournament).
Tournament should have Many-To-One Association (target: Event).
Tournament should have One-To-Many Association (target: Schedule).
Schedule should have Many-To-One Association (target: Tournament).

Section is common name for “Tournament”, “Official broadcast” etc.
“Event” object is available from current page attribute.

First you should list each Section in separate list and filter it by “Event” object.
So “Tournament” has its own list, “Official broadcast” another and so on.

Next, when you fetch $tournaments (or any other Section list), you want to do foreach loop and then inside make another list
where you filter “Schedules” by “Tournament” object (or any other Section object).

That’s all.
And yes, if you want to use Express List block, then you need to add as many of them as there are Sections and skip first part.

---

If you want to do it in “one query”, you have to add another association.
Event should have One-To-Many Association (target: Schedule).
Schedule should have Many-To-One Association (target: Event).

This way you can filter “Schedule” list by current “Event” and then reorganize results in PHP.

Drawback is that you will have “duplicated” information about associated event (in normalized database, you can access “Event” by this chain of relations: “Schedule → Tournament → Event”).
But this kind of denormalization is fine, if it helps speed up database.
I would just add custom Express validator when adding “Schedule”, to check if “Event” is matching selected “Tournament → Event” relation.

---

Alternative way is to add sql “JOINS” manually, but it’s rather hard if you don’t know SQL well.
Concrete Express API doesn’t have many methods similar to filterByAssociatedEntry(), so you would need to write queries manually.

---

Though in the end, it probably doesn’t matter how you make “Event” page, since it will usually contain a limited number of elements.
You will encounter first real problems when you make “Search” page with filters, pagination and all the bells and whistles.

[BloodyIron]

Alright well I’ll have to weigh the options and decide.

There is one part that I am not yet sure on what to do, and I roughly asked about it earlier.

When creating a new $list, are you aware of a way that I can filter (out) based on whether an Association has an Entry selected at all? (boolean true/false kind of thing) and then stuff the results matching “true” into a new $list2 or whatever?

Lots to think about here, thanks again!

[Parasek]

If you run through list with foreach loop then something like that:

use Concrete\Core\Entity\Express\Entry;
use Concrete\Core\Express\EntryList;
use Concrete\Core\Support\Facade\Express;

$entity = Express::getObjectByHandle('tournament');
$association = $entity->getAssociation('schedules');

$list = new EntryList($entity);
$entries = $list->getResults();
/** @var Entry[] $entries */
foreach ($entries as $entry) {
    $entryAssociations = $entry->getEntryAssociation($association);
    if ($entryAssociations === null) {
        // no associations
        echo 'none<br/>';
    } else {
        $entries = $entryAssociations->getSelectedEntries();
        echo count($entries).'<br/>';
    }
}

Alternatively you can build sql using query builder etc.

[BloodyIron]

Okay I think I’m going to take all this and see what I can bake up. Thanks!

[BloodyIron]

Hey so one thing I’m stuck with… for the “Event” Page Attribute (Express Entry Select)… I’m not seeing a way to get the EntryID when referencing c$->getAttribute(‘name_of_express_attribute’);, and this is breaking my attempted usage of ->filterByAssociatedEntry(…);

What would you recommend I try to get the Entry ID? That way I can substantiate “$whatever = Express::getEntry(#);”

Thanks!

[Parasek]

/** @var \Concrete\Core\Entity\Express\Entry $entry */
/** @var \Concrete\Core\Entity\Attribute\Value\Value\ExpressValue $expressAttribute */
$expressAttribute = $c->getAttribute('event_express_attribute');
if ($expressAttribute) {
    $selectedEntries = $expressAttribute->getSelectedEntries();
    if (is_object($selectedEntries)) {
        $entry = $selectedEntries[0];
        dd($entry);
    }
}