The content of this section will cover the part of new entities/relations creation specific to MooseQuery. It will not cover how to create new Moose entities. For this, please refer to Moose documentation.
Developer Documentation
Introduction
MooseQuery is the result of the Inria research team RMoD. This page documents the internals of MooseQuery. As prerequisites, it supposes the reader has a minimum of knowledge of Fame and Famix. If not, please read the Moose documentation, if you wish to understand everything.
This documentation covers:
- How to complete a Famix meta-model to make new entities/relations queryable via MooseQuery
- How the containment tree exploration described in the user documentation works
- How the navigation queries works
- How the scope queries works
The first section will be useful if you wish to extend/create a meta-model. The three next sections only describe how the query system works. It does not requires any change from the developers of a meta-model.
Completion of the Meta-model
First, the developer of the model has to make his new entities/relations queryable with MooseQuery.
Relations definitions
As said in the user documentation, MooseQuery is based on two concepts:
- Imbrication of entities: which entity contains which entities.
- Associations between the entities: how the entities interact with each other.
Here, for example, Entity1 is the container of Entity2 and Entity1 is the source of an association whose Entity3 is the target.
Moose query needs to know those relations in order to work. To declare them you just need to use three pragma: <container>, <source> and <target>
The container pragma should be put in the contained entity in the method defined to access the container.
At the meta-model level, an entity may have several parents. For example, the container of a class can be a package or a method in the case of inner class. In that case, both methods #parentPackage and #container must have the pragma.
Containment Relation Definition
FAMIXAttribute>>parentType
<MSEProperty: #parentType type: #FAMIXType opposite: #attributes>
<MSEComment: 'Type declaring the attribute. belongsTo implementation'>
<container> "<=== Here is the containment relation definition"
^ parentType
The source and target pragmas must be in the methods relevant of the association.
Association Definition
FAMIXAccess>>accessor
<MSEProperty: #accessor type: #FAMIXBehaviouralEntity opposite: #accesses>
<MSEComment: 'Behavioural entity making the access to the variable. from-side of the association'>
<source> "<=== Here is the association's source definition"
^ accessor
FAMIXAccess>>variable
<MSEProperty: #variable type: #FAMIXStructuralEntity opposite: #incomingAccesses>
<MSEComment: 'Variable accessed. to-side of the association'>
<target> "<=== Here is the association's target definition"
^ variable
Traits usage
Relations definitions (via the pragmas) give to MooseQuery all the requirements for querying the meta-model. Now, the new entities of the model need to have the behaviour, i.e., the methods, to execute the queries. To do so, MooseQuery comes with a set of Traits to use in the new entities.
| Trait | Default users | Description |
|---|---|---|
| TAssociationMetaLevelDependency | FAMIXAssociation | This trait includes the information about associations needed for the navigation queries. It needs to be used by every association in a meta-model. |
| TEntityMetaLevelDependency | FAMIXNamedEntity | This trait includes the information about entities and adds the behaviour to explore the containment tree and execute scope queries. It needs to be used in every entity's hierarchy in a meta-model. |
| TDependencyQueries | FAMIXNamedEntity | This trait includes the information about entities and adds the behaviour to execute navigation queries. It needs to be used in every entity's hierarchy in a meta-model. |
| TOODependencyQueries | FAMIXContainerEntity | This trait includes the information about object-oriented entities and adds the behaviour to execute navigation queries. It needs to be used in every entity's hierarchy in a meta-model instead of TDependencyQueries. |
All those traits need to be used when implementing a new meta-model. For performance reasons, some more work is needed when using those traits. Some methods exist in two versions. One private with the real behaviour, and the other public that access a class instance variable or, if nil, set it with the result of the private method. Due to the fact that traits are currently stateless in Pharo, all the public methods have to be implemented for your new metaclass using directly the traits and not by inheritance. For example, FAMIXAnnotationInstance does not inherit from FAMIXNamedEntity but can be queried similarly. The same occur for new links using TAssociationMetaLevelDependency without inheriting from FAMIXAssociation. Here is the list of caches to implement:
-
TAssociationMetaLevelDependency class
- #sourceTypes
- #targetTypes
-
TEntityMetaLevelDependency class
- #allChildrenTypes
- #allParentTypes
- #childrenSelector
- #parentSelector
- #allIncomingAssociationTypes
- #allOutgoingAssociationTypes
- #incomingMSEProperties
- #outgoingMSEPreperties
Example of Cache Implementation
"Already defined in MooseQuery:"
TEntityMetaLevelDependency class>>allChildrenTypes
^ self explicitRequirement
TEntityMetaLevelDependency class>>privateAllChildrenTypes
^ (self childrenTypes withDeepCollect: #childrenTypes as: Set) asOrderedCollection
"Method to implement:"
FAMIXNamedEntity class>>allChildrenTypes
^ allChildrenTypes ifNil: [ allChildrenTypes := self privateAllChildrenTypes ]
The model builds caches on classes to speed up queries. It means that during the development of the meta-model it is often required to reset those caches.
In order to reset the caches you can execute:
MooseModel resetMeta
This method will call #resetMooseQueryCaches on all subclasses of FAMIXEntity. If you add new caches (in a class that directly uses the trait and not through inheritance like FAMIXAnnotationInstance), you need to override in the class side of your entity #resetMooseQueryCaches to reset the added caches.
Example of Cache Reset
FAMIXNamedEntity class>>resetMooseQueryCaches
super resetMooseQueryCaches.
childrenSelectors := parentSelectors := allChildrenTypes := allParentTypes := outgoingMSEProperties := incomingMSEProperties := incomingAssociationTypes := outgoingAssociationTypes := nil
Containment Tree Exploration
This section and the next ones explains how works MooseQuery's queries. From this point, no more action is required from the developer to have working queries on its meta-model.
Now that we saw how to make entities queryable, this documentation will focus on explaining the internal mechanisms of MooseQuery. The easiest queries are the ones related to the exploration of the containment tree.
MooseQuery allows querying the children/parents of an entity. This behaviour is based on the meta-model of the application. Each Moose entity possesses a meta-description that lists all the properties of the entity. These properties can be obtained with:
Getting the Properties of an Entity
MooseEntity class>>allDeclaredProperties
"All properties described in the meta-model"
^self mooseDescription allAttributes
FAMIXType allDeclaredProperties. "==> An array of property descriptions (See screenshot bellow)"
Each property knows:
- If it is a property enabling to access the contained entities (via the method #isChildrenProperty)
- If it is a property defining a parent entity (via the method #isContainer)
- The selector returning the property in the entity's API (via the method #implementingSelector)
So, based on the meta-model and the moose pragma, it is possible for a Famix class to get the selectors to access its parents or children entity using the #parentSelectors and #childrenSelectors methods respectively. Here again a cache mechanism is used.
Getting the parents/children selectors of an entity
"Implementation details:"
TEntityMetaLevelDependency class>>privateChildrenSelectors
^ self allDeclaredProperties select: #isChildrenProperty thenCollect: #implementingSelector
FAMIXNamedEntity class>>childrenSelectors
^ childrenSelectors ifNil: [ childrenSelectors := self privateChildrenSelectors ]
TEntityMetaLevelDependency class>privateParentSelectors
^ self allDeclaredProperties select: #isContainer thenCollect: #implementingSelector
FAMIXNamedEntity class>>parentSelectors
^ parentSelectors ifNil: [ parentSelectors := self privateParentSelectors ]
"Example of use:"
FAMIXType parentSelectors. "=> #(#container #parentPackage)"
FAMIXType childrenSelectors. "=> #(#annotationInstances #types #definedAnnotationTypes #attributes #functions #methods)"
So, whatever the selector used in the meta-model to access the contained elements of a given entity, it is possible to access them through the children method that dynamically call each selector returned by the #childrenSelectors methods. The same is possible to get the parents of a given entity. Once we retrieved all parents/children selectors it is possible to perform them on an entity to get their results and gather the children/parents of this entity.
Getting the parents/children of an entity
"Implementation details:"
TEntityMetaLevelDependency>>children
| res |
res := OrderedCollection new.
"Since some selector can return nil, an entity or a collection of entities, we need to do a nil check and to ensure everything is a collection via #asCollection"
self childrenSelectors do: [ :accessor | (self perform: accessor) ifNotNil: [ :r | res addAll: r asCollection ] ].
^ res asSet
TEntityMetaLevelDependency>>parents
| res |
res := OrderedCollection new.
"Since some selector can return nil, an entity or a collection of entities, we need to do a nil check and to ensure everything is a collection via #asCollection"
self parentSelectors do: [ :accessor | (self perform: accessor) ifNotNil: [ :r | res addAll: r asCollection ] ].
^ res asSet
"Example from the user documentation:"
package1 children. "=> { package2 . class1 }"
class3 children. "=> { attribute1 . attribute2 }"
package1 parents. "=> { }"
class3 parents. "=> { package2 }"
class4 parents. "=> { package3 . namespace1 }"
No risk to forget a selector, it is dynamically computed from the meta-model.
Similarly, it is possible to get children/parents recursively. Since containment trees can be pretty big, we use an accumulator to avoid creating a lot of collections.
Getting the parents/children of an entity recursively
"Implementation details:"
TEntityMetaLevelDependency>>allChildren
"Returns all the children and sub-children of an entity, i.e my children and those of my children, and those of the children of my children, etc"
^ self addAllChildrenIn: OrderedCollection new
TEntityMetaLevelDependency>>addAllChildrenIn: aCollection
aCollection addAll: self children.
self children do: [ :each | each addAllChildrenIn: aCollection ].
^ aCollection
"Examples from the user documentation:"
package1 allChildren. "=> { package2 . class1 . class2 . class3 . attribute1 . attribute2 }"
class3 allChildren. "=> { attribute1 . attribute2 }"
class3 allParents. "=> { package2 . package1 }"
attribute1 allParents. "=> { class3 . package2 . package1 }"
Navigation Queries
This section will explain how the navigation queries works.
Navigation queries, as explained in the user documentation, are parametrized by:
- The direction of the query. Should the receiver be the source or the target of the associations to gather?
- The scope of the query. Should the query deal with the receiver or deal with the receiver and its children.
- The association(s) to query. It is possible to query a specific association or all the associations.
All those parameters will be explained in this section.
To launch a navigation query, all Moose entities can use an API provided by the trait TDependencyQueries, as explained previously. Those methods all begin in the same way. They send the request to a new instance of MooseQueryCalculator parametrized with the direction of the query.
Management of the queries direction
This section covers the query direction management.
As shown in the UML diagram above, directions are managed via a Design Pattern strategy. A MooseQueryCalculator refers to a strategy that will be either a MooseQueryIncomingDirectionStrategy or a MooseQueryOutgoingDirectionStrategy. The strategies manage everything related to the direction of the query. This includes:
- Obtaining the possible association types for a given entity depending on the direction
- Obtaining the meta-properties of an entity defining a relation of source/target between the entity and an association
- Creating a MooseQueryResult for the right direction
Example of Responsibility Dispatch Between Strategies
MooseQueryAbstractDirectionStrategy class>>associationTypesFor: anEntity
^ self subclassResponsibility
MooseQueryIncomingDirectionStrategy class>>associationTypesFor: anEntity
^ anEntity incomingAssociationTypes
MooseQueryOutgoingDirectionStrategy class>>associationTypesFor: anEntity
^ anEntity outgoingAssociationTypes
The selection of the right strategy is done via the symbol #in or #out given by the user. The MooseQueryAbstractDirectionStrategy is able to return the right strategy with this symbol as input.
Strategy Selection Mechanism
MooseQueryAbstractDirectionStrategy class>>fromSymbol: aSymbol
"I return the right strategy from a Symbol representing a direction"
^ self allSubclasses detect: [ :each | each determinatingSymbol = aSymbol ] ifNone: [ self error: 'The symbol passed to the query should be either #in or #out. Here we got ' , aSymbol asString ]
MooseQueryIncomingDirectionStrategy class>>determinatingSymbol
^ #in
MooseQueryOutgoingDirectionStrategy class>>determinatingSymbol
^ #out
Local queries
Now that it is clear how the distinction between the directions of the query is managed, we can see how the navigation queries works. Let's begin with the local queries.
Local navigation queries focus only on an entity and do not care about its children. Considering that we want to compute the incoming references of a class. With local queries, we don't care if the class contains inner classes that are referenced or not. It will not gather associations of its children.
It is possible to launch a local query via the selector TDependencyQueries>>#queryLocal:with: taking as parameter the direction of the query and the studied association class. This method creates a new MooseQueryCalculator parametrized with the right direction strategy and call MooseQueryCalculator>>#queryLocal:for: with the association class and the receiving entity as parameters.
MooseQueryCalculator>>#queryLocal:for: gather the associations and creates a new MooseQueryResult with them via the method #queryResultOn:with:. This method, sent to MooseQueryIncomingDirectionStrategy, return a MooseIncomingQueryResult for the entity. Sent to MooseQueryOutgoingDirectionStrategy, return a MooseOutgoingQueryResult. This is shown in the script bellow.
MooseQueryCalculator>>#queryLocal:for:
MooseQueryCalculator>>queryLocal: aFAMIXClassAssociation for: anEntity
^ self strategy queryResultOn: anEntity with: (self queryLocal: aFAMIXClassAssociation for: anEntity in: OrderedCollection new) asSet
To gather the associations we get all the properties returning associations in the right direction from the direction strategy. We then perform all the property names on the entity and select the ones which type match the one as parameter.
MooseQueryCalculator>>#queryLocal:for:in:
MooseQueryCalculator>>queryLocal: aFAMIXClassAssociation for: anEntity in: aCollection
"The goal here is to get all the entity's direct associations of the class in parameter. This will not check the associations of the entity's children."
| properties |
"Instead of #to:do: we could just use #do: but this implementation is much faster. Maybe sista will remove the needs of the todo later."
1 to: (properties := self strategy msePropertiesOf: anEntity) size do: [ :ind | (anEntity perform: (properties at: ind) name) ifNotNil: [ :coll | aCollection addAll: (coll asCollection select: [ :each | each isKindOf: aFAMIXClassAssociation ]) ] ].
^ aCollection
The code of this method is more complex than it should be because of performance reasons. This method can be called a lot in certain cases and it needs to be as efficient as possible. A more readable but less efficient version could be:
MooseQueryCalculator>>#queryLocal:for:in:, readable version
MooseQueryCalculator>>queryLocal: aFAMIXClassAssociation for: anEntity in: aCollection
"The goal here is to get all the entity's direct associations of the class in parameter. This will not check the associations of the entity's children."
(self strategy msePropertiesOf: anEntity) do: [ :property | (anEntity perform: property name) ifNotNil: [ :coll | aCollection addAll: (coll asCollection select: [ :each | each isKindOf: aFAMIXClassAssociation ]) ] ].
^ aCollection
Global queries
Local queries are useful, but most of the time users want to get all the associations taking into account the entity itself as source (resp. target) or its contained entities. For example it can be relevant to know all the incoming invocations of class1 (cf. user documentation examples). In this case, a class cannot be the target of an invocation, but what we want to get are all the invocations targeting any methods contained in the class.
These queries begin in the same way as local navigation queries. The entity will create a new MooseQueryCalculator parametrized with a direction strategy and return a new MooseQueryResult with the resulting associations. The only part changing is the way of the resulting associations are computed.
It will begin by computing the associations of the receiver via the same method as the local navigation queries (MooseQueryCalculator>>queryLocal:for:in:), and then iterate on the children of the entity (which is possible via the methods of the containment tree exploration) to call recursively the query.
MooseQueryCalculator>>#query:for:in:
MooseQueryCalculator>>query: aFAMIXClassAssociation for: anEntity in: aCollection
"I know that we could use #do: here, but this algo takes too long on big model, so this is a speed up while the sista vm is not ready for production"
| selectors |
self queryLocal: aFAMIXClassAssociation for: anEntity in: aCollection.
1 to: (selectors := anEntity childrenSelectors) size do: [ :index |
(anEntity perform: (selectors at: index))
ifNotNil: [ :children |
| coll |
1 to: (coll := children asCollection) size do: [ :i | self query: aFAMIXClassAssociation for: (coll at: i) in: aCollection ] ] ].
^ aCollection
As for the local queries, the readability of the method is sacrificed for performances. Find below a more readable version.
If we compute MooseQueryCalculator query: FAMIXInvocation for: class1, MooseQueryCalculator queryLocal: FAMIXInvocation for: class1 returns an empty collection since aFAMIXClass cannot be the target of an invocation. Then for each child of this entity the search is done. Concretely, it is compute only for the methods of class1 or the methods of any other entity contained recursively in class1.
MooseQueryCalculator>>#queryLocal:for:in:, readable version
MooseQueryCalculator>>query: aFAMIXClassAssociation for: anEntity in: aCollection
self queryLocal: aFAMIXClassAssociation for: anEntity in: aCollection.
anEntity childrenSelectors
do: [ :selector | (anEntity perform: selector) ifNotNil: [ :children | children asCollection do: [ :child | self query: aFAMIXClassAssociation for: child in: aCollection ] ] ].
^ aCollection
Queries on all associations
One use of navigation queries is to find the dependencies between entities. To find all the dependencies of an entity it is necessary to compute all associations.
MooseQuery allows to directly compute all the associations incoming or outgoing of an entity at once. For performance reasons, the implementation visit the containment tree on time instead of iterating on all associations to execute the queries.
The main change in the implementation is done in the local query. Instead of selecting the meta-property where the implementing type is a specific association class, it will gather the result of all properties defining an incoming/outgoing relation with the entity.
MooseQueryCalculator>>#queryAllLocalFor:in:
MooseQueryCalculator>>queryAllLocalFor: anEntity in: aCollection
| properties |
"Instead of #to:do: we could just use #do: but this implementation is much faster. Maybe sista will remove the needs of the todo later."
1 to: (properties := self strategy msePropertiesOf: anEntity) size do: [ :ind | (anEntity perform: (properties at: ind) name) ifNotNil: [ :coll | aCollection addAll: coll asCollection ] ].
^ aCollection
Scope Queries
This section explains how the scope queries works.
A scope query is parametrized by:
-
The direction of the query
- Up in the containment tree (#atScope:, #allAtScope:)
- Down in the containment tree (#toScope:, #allToScope:)
- Both up and down in the containment tree (#withScope:, #allWithScope:)
- The Famix entity class defining the scope to query
A scope query can be applied on:
- A Famix entity
- A MooseQueryResult obtained from a navigation query
Query on a Famix entity
Queries going up or down in the containment tree for the scope search are usable respectively via the methods TEntityMetaLevelDependency>>#atScope: and #toScope:. For example we can want all the incoming invocations of package P2 (cf. user documentation examples) at class scope. Meaning that we want to compute all the invocations that reach a method recursively contained in P2. But, as a result, we don't want neither the invocations nor the target methods of these invocations. We want the classes containing these methods.
These methods begin to create a collection to store the results and call #atScope:in:/#toScope:in: with the newly created collection as a parameter. This is done for performance reasons. Instead of creating multiple collections and use concatenation on them, we create one at the beginning and pass it as a parameter of the query methods.
The query then checks if the receiver is of the searched kind. In case it is, the query end and returns the receiver as result. If it is not the case then the query is repeated on the parents/children of the entity depending on the direction in the containment tree of the query.
TEntityMetaLevelDependency>>#atScope:in:
TEntityMetaLevelDependency>>atScope: aClassFAMIX in: aCollection
(self isKindOf: aClassFAMIX)
ifTrue: [ aCollection add: self ]
ifFalse: [ "The content of this block could be much more readable with #do: but we do this solution for performances... We need this method to be really really performant."
| selectors |
1 to: (selectors := self parentSelectors) size do: [ :ind | (self perform: (selectors at: ind)) atScope: aClassFAMIX in: aCollection ] ].
^ aCollection
As in the navigation queries, this code could be more readable but it is the way it is for performance reasons. Find the more readable code below.
TEntityMetaLevelDependency>>#atScope:in:, readable version
TEntityMetaLevelDependency>>atScope: aClassFAMIX in: aCollection
(self isKindOf: aClassFAMIX)
ifTrue: [ aCollection add: self ]
ifFalse: [ self parentSelectors do: [ :selector | (self perform: selector) atScope: aClassFAMIX in: aCollection ] ].
^ aCollection
In the case of the up and down queries, we will just check if the entity can do up/down scopes and apply them if needed.
TEntityMetaLevelDependency>>#withScope:in:
TEntityMetaLevelDependency>>withScope: aClassFAMIX in: aCollection
self allParentTypes detect: [ :class | aClassFAMIX = class or: [ aClassFAMIX inheritsFrom: class ] ] ifFound: [ self atScope: aClassFAMIX in: aCollection ].
self allChildrenTypes detect: [ :class | aClassFAMIX = class or: [ aClassFAMIX inheritsFrom: class ] ] ifFound: [ self toScope: aClassFAMIX in: aCollection ].
^ aCollection
The recursive version (#allAtScope:, #allToScope: and #allWithScope:) is approximately the same, except it will not stop if an entity of the right kind is found.
TEntityMetaLevelDependency>>#allAtScope:in:
TEntityMetaLevelDependency>>allAtScope: aClassFAMIX in: aCollection
| selectors |
(self isKindOf: aClassFAMIX) ifTrue: [ aCollection add: self ].
"The content of this block could be much more readable with #do: but we do this solution for performances... We need this method to be really really performant."
1 to: (selectors := self parentSelectors) size do: [ :ind | (self perform: (selectors at: ind)) allAtScope: aClassFAMIX in: aCollection ].
^ aCollection
Query on a MooseQueryResult
Once we understand the scope queries applied to entities, it is easy to understand the scope queries on MooseQueryResult.
MooseQueryResults store the associations gathered via a navigation query. Applying a scope query on it gathers the opposites of the first query receiver and apply the scope query on each of them.
TDependencyQueryResult>>#atScope:
TDependencyQueryResult>>atScope: aClassFamix
^ self newObjectResultWith: (self storage inject: OrderedCollection new into: [ :res :dep | (self opposite: dep) atScope: aClassFamix in: res ]) asSet