Why isn’t That Cron Task Running!

All of them are running, running and running, the long, never ending marathon but what is it with that one lone runner, why isn’t he running?

It’s a rare occurrence but you may come across such cron task that is sitting there not executing at all while all the other cron tasks are running just fine! You would see that though it is set to active and the history logs tell you that it started successfully or it was working fine till a certain point but after that there hasn’t been any action.

What is it with this cron task?

It cannot be the admin mode since other cron tasks are running fine. It could be the mxe.crontask.donotrun property but in that case it wouldn’t have started or actioned at all. But better to still double check.

Rare but not impossible, this may be caused by incorrect entry in the task scheduler table.

What is Task Scheduler Table?

The TASKSCHEDULER table is used by Maximo to control the schedule of cron tasks.

Before a cron task runs, Maximo gets the last run information from TASKSCHEDULER table, and if no information is found, it will insert a new record in TASKSCHEDULER table if needed.

For more information about task scheduler table check this post.

What’s the issue with the Task Scheduler?

As I mentioned earlier, the issue may be caused by incorrect entry in the task scheduler table, especially when the LASTRUN is greater than the LASTEND which means the cron task is running or atleast the system thinks that the cron task is running and wouldn’t invoke it again.

TaskScheduler

This could be because the JVM crashed while the cron task was running and the execution couldn’t complete and so the last run was not updated. It is also possible that the cron task actually got into a hung state during execution.

Another possibility is that there are two entries for the same cron task in the task scheduler table which confuses the system. May be the cron task is running too quickly and it sends an entry twice to the task scheduler, because it didn’t have enough time to complete the first execution. Cron task running on multiple JVM in a clustered environment could also be the cause.

How to fix it?

Delete the entries for the cron task from the task scheduler table. The cron task manager would see that the cron task is active but doesn’t have entry in the taskscheduler so it would create an entry and wait till the next schedule and to run the cron task.

If you want to understand the sequence of events that happens when the cron task starts and shuts down, you can check this post.

Until next time!

Cheers!

Data Restrictions and their impact on MBOSets

At some point in time every maximo developer or support engineer would have implemented data restrictions to hide data or make it un-editable for the users. Since they are fairly easy to implement and only require a sign out and sign in for them to take effect for a user, they are very widely used. However, if not implemented carefully, they can seriously impact performance.

I am going to explain how mbosets are impacted by data restrictions, the qualified type of data restriction in particular.

When a qualified type of data restriction is implemented on an object with a condition, the user will have access to only those records of that object which are fetched by the condition’s where clause i.e. those records which qualify the condition. This clause is applied over and above any exist clause.

When it comes to java customization, the impact of data restrictions on the mboset depend on the way the mboset has been fetched in the code. If the mboset has been fetched through MXServer then the data restrictions will apply (if you remember we pass the userinfo to fetch the set) but if the set is fetched through a relationship then the data restrictions don’t apply.

You may ask why this different behavior? That’s because when an mboset is fetched through relationship, it becomes a child mboset and the rule is – owner must have access to all it children. That’s why!

Let me explain this with an example. Let’s say there is a data restriction on the asset object to show only those assets that have their type as FACILITIES. When a user navigated to the asset application he would be shown only FACILITIES type assets. When he navigates to the subassembly section of any of these assets, he would be shown all child assets of that asset which may or may not be of the type FACILITIES. Seems justifying right!

Note: Subassembly section in assets is a relationship of asset with itself to show child assets.

2016-11-15-20_38_48-security-groups

2016-11-15-20_42_39-assets

2016-11-15-20_42_49-assets

Let see how and where does this impact –

Domains – Table domains are fetched using MXServer so data restrictions apply. But if you are creating a table domain through code and in the getList method you use relationship then data restriction wont apply.

Dialogs – Dialogs are mostly created using a relationship so data restrictions would not apply. In some cases, mboname is used instead of relationship, in that case data restrictions would apply.

Non-Persistent Fields – Non-persistent fields are populated through java code. If the code uses MXServer to fetch some data and populate the field, the non-persistent field might not get populated or might display incorrect data if data restriction exist on that fetched set.

Other Customizations – Certain logic may fail due to data restriction if these logics are not implemented keeping data restrictions in mind and specially if MXServer is used to implement these logics.

Developer must keep in mind the impact of data restrictions while implementing logics. I am not saying MXServer is a strict no-no, but if data restrictions are going to be placed the impact must be considered.

On the good side even if qualified data restrictions are applied, they are majorly applied on main objects isn’t so!

Cheers!

Conditionally Changing the Lookup on an Attribute

Whenever I travel, I either read or I write. These days I am travelling a lot and I am on a writing spree, so you would be seeing quite a few posts from me.

That brings me to this unique requirement to display different lookups on a field based on a condition. A simple example of this is displaying a location lookup on a field if, lets say the orgid is EAGLENA else display an asset lookup on the same field.

This can be achieved through conditional UI in application designer.

The images below show how to setup this through conditional UI on a field EQ1 on ASSET.

2016-11-19-14_34_39-application-designer

2016-11-19-14_32_21-application-designer

2016-11-19-14_32_28-application-designer

2016-11-19-14_39_46-assets

2016-11-19-14_39_55-assets

2016-11-19-14_40_22-assets

2016-11-19-14_40_25-assets

Have a great day!

Recursive Queries, Maximo Lookups and Deadlocks

There are several scenarios where you have a requirement to present data that cannot be queried through a simple select statement but requires looping through the table as the data resides in a parent child hierarchy in the table. Work order hierarchy is one such example where a work order record has its parent listed in one of its column, and the parent’s record would have its own parent listed in its record and so on. So to find all the child work orders to the leaf level under a work order, looping would be required.

Note: Although ANCESTOR tables maintain the complete ancestor hierarchy but at times they can have incorrect data.

This is typically achieved in a relational database through a recursive query and the traditional way is to use CTE or Common Table Expression which uses the ‘WITH’ statement to construct recursive queries. I will not go into details of how to construct or use CTE, their detailed documentation is readily available online.

Oracle has introduced a much simpler syntax to execute recursive queries – the START WITH, CONNECT BY statement. It has a lot of limitation compared to the traditional CTE method but for simple querying its gets the work done in a lot less effort. The START WITH, CONNECT BY statement can also be used in DB2 by enabling the oracle compatibility mode 08.

When it comes to table domains in Maximo, using the traditional CTE is a challenge since in table domains only the where clause can be specified and querying using CTE has an entirely different syntax so it cannot be specified in the listwhereclause. The START WITH, CONNECT BY on the contrary, has the syntax similar to the traditional select statement and can be used in the table domains to query data. Here is an example of the START WITH, CONNECT BY statement that lists all the child work orders of the work order WO-1990 till the leaf level.

SELECT * FROM WORKORDER
START WITH WONUM = ‘WO-1990’
CONNECT BY PRIOR WONUM = PARENT AND ISTAKS = 0;

A little bit about table domains, when you open a lookup in Maximo which displays data through table domain, it doesn’t fetch the entire result set in one go but only that no. of rows that are displayed on the first page of the lookup. When you click the next page on the lookup, the next set of records is fetched and so on. What that means is if the lookup is configured to display only 10 rows at a time, when the lookup is opened, only first 10 records are fetched from the database, now if the next button on the lookup is clicked, the next 10 records are fetched and so on.

This is where it gets interesting, what I have observed is if you use the START WITH, CONNECT BY statement in a domain, when the lookup for this domain is opened, it holds a lock on the table(row level lock) till the time that lookup is closed. This is not the case with non-recursive queries, they never hold a lock, even complex queries with joins and sub-selects.

This is where the problem lies, when the lookup with the START WITH, CONNECT BY statement is opened in one application and another user or the same user in another application tries to do a transaction (update or delete) on the same set of rows as queried by the lookup, the user doing the transaction is presented with a deadlock error. Overriding the isolation level for the recursive query also doesn’t help.

db_error_911

The image above shows an error presented to the user in a similar scenario but in this case the lookup and the transaction performed was on the LOCHIERARCHY object.

So it is better to avoid using START WITH, CONNECT BY statement in lookups and use alternate solutions like ANCESTOR tables or views to display data. Some of the lookups can be tricky and may require lot of effort to avoid using START WITH, CONNECT BY statement but if there are deadlock issues, it’s worth the effort.

Cheers!

Import Export Multiple XMLs in Maximo

Did you know that maximo support multi XML import export?

If you want to export a bunch of application xmls from one environment to another or may be you want to edit more than one xml and then import them back into maximo, this functionality can come in handy.

To use this functionality, filter the list of applications to be exported on the list tab of the application designer and click the export button on the toolbar. Maximo will export a single xml containing all the applications filtered on the list tab. This xml can be imported back in the same environment or other environments as per your need.

The screenshot below shows 3 applications being exported. The exported xml shows 1 expanded application presentation and other 2 collapsed.

2016-04-05 21_39_08-Application Designer.png

2016-04-05 21_38_59-eamserver_9080_maximo_ui_presentationset.xml_event=exportall&uisessionid=3&csrft.png

Adding custom attribute to asset template and copying it to the asset

When it comes to adding a custom attribute to asset template and getting it copied to the asset, one might think that this may require extending the asset template class and adding the code to get the field copied or probably configuring a cross over domain. But its way easier than that!

Just creating the custom attribute in asset template and in asset with the same name gets the job done. When the asset is created from the asset template, the data from the custom field in asset template gets copied over to the field with the same name in asset. I did a quick check on this and it works as expected. Have a look –

2016-02-14 10_52_35-Asset Templates.png

2016-02-14 10_55_09-Asset Templates.png

2016-02-14 10_55_44-Assets.png

How Workflow Conditions are Evaluated

While looking at the debug logs of Maximo I noticed queries fired against the dummy_table. I wondered what is it that Maximo needs to query against the dummy_table. After looking at the sql and some digging in, I found that Maximo uses the dummy_table for evaluating the conditions.

The condition of a condition node in a workflow is evaluated in 2 ways –

  1. The condition is validated using the Parser (standard way, on the object).
  2. If the parser fails, the condition is validated using SqlFormat on dummy_table.

To explain it further, consider a workflow on the PO having one of the following conditions-

  1. Condition: poid=:poid -this will pass the Parser, so validation would be successful.
  2. Condition: exists (select * from ADDRESS where orgid = :orgid and addresscode = :shipto) – this will fail the parser, but will pass the SqlFormat checking on dummy_table using this sql: Select count(*) from dummy_table where (exists (select * from address where orgid =  ‘org01’ and addresscode =  ‘add01’));

So validation would be successful.

  1. Condition: poid = :poid and exists (select * from ADDRESS where orgid = :orgid and addresscode = :shipto) will fail the parser, will also fail the SqlFormat check because dummy_table doesn’t have a column named poid :

sql: Select count(*) from dummy_table  where  poid =  32767 and exists (select * from address where orgid =  ‘org01’  and addresscode =  ‘add01’ );

So validation would fail.

In this case the poid = :poid doesn’t really make any sense in the sql.

So while writing a condition make sure that you are not writing a sql that would fail.

The Significance of “Independent of Other Groups” in Security Groups

The “Independent of Other Groups” checkbox in the security group application is one of the most misunderstood concepts in Maximo. What is the significance of this checkbox? How does it affect the security authorization of the group? Let’s have a look.

IndependentGroup

As the name suggests, the “Independent of Other Groups” checkbox on the main tab of the security group application specifies weather the group is independent of other groups or not. What that means is whether the authorization of this group can be merged with the authorizations of other groups. This is only applies to multisite implementations, for single site implementation this checkbox doesn’t have any significance. Lets understand how this works –

Scenario 1: Consider a multisite implementation; the user belongs to the 2 security groups with the following privileges:

Group 1: Site A and Work Order Tracking application

Group 2: Site B and Purchase Orders application

If these security groups are both independent then the user ends up with rights for Work Order Tracking on Site A and Purchase Orders on Site B. If however the security groups are non-independent then the privileges combine and the user ends up with both Work Order Tracking and Purchase Orders on Sites A and B. Basically you sum up the privileges and if any of them overlap you take the highest level.

Scenario 2: Consider a single site implementation; the user belongs to the 2 security groups with the following privileges:

Group 1: Site A (or all sites authorization) and Work Order Tracking application

Group 2: Site A (or all sites authorization) and Purchase Orders application

Since there is only one site, irrespective of whether these groups are independent or non-independent the user will have the rights to both Work Order Tracking and Purchase Orders on Sites A.

Scenario 3: Consider a single site implementation; the user belongs to the 2 security groups with the following privileges:

Group 1: NO SITE and Work Order Tracking application

Group 2: Site A (or all sites authorization) and Purchase Orders application

If these security groups are both independent then the user ends up with rights for only Purchase Orders on Site A. This is because the two groups are independent and group 1 doesn’t have authorization to any site hence user doesn’t have rights to Work Order Tracking in Site A. If however the security groups are non-independent then the privileges combine and the user ends up with both Work Order Tracking and Purchase Orders on Sites A.

NOTE: This is why the independent checkbox should not be checked for single site implementations.

Scenario 4: Consider a multisite implementation; the user belongs to the 2 security groups with the following privileges:

Group 1: Site A and Work Order Tracking application – Read, Route workflow Access

Group 2: Site B and Work Order Tracking application – Read, Change Status Access

If these security groups are both independent then the user ends up with rights for Work Order Tracking – Read, Route workflow Access on Site A and Work Order Tracking application – Read, Change Status Access on Site B. If however the security groups are non-independent then the privileges combine and the user ends up with Work Order Tracking – Read, Route workflow & Change Status Access on Sites A and B.

Scenario 5: Consider a multisite implementation; the user belongs to the 2 security groups with the following privileges:

Group 1: Site A and PO Limit of 10,000

Group 2: Site B and PO Limit of 50,000

If these security groups are both independent then the user ends up with PO Limit of 10,000 for Site A and PO Limit of 50,000 for Site B. If however the security groups are non-independent then the privileges combine and the user ends up PO Limit of 50,000 for both Sites A and B.

Note: Never change the MAXEVERYONE group to independent group as this group has a lot of conditional grants which will stop working if this group is made independent without granting access to all sites (similar to Scenario 3).

IBM Tech Note on why MAXEVERYONE shouldn’t be set as independent

How to detect and resolve database connection leak?

There are 2 very detailed posts on IBM DeveloperWorks written by Manjunath about database connection leaks – how to detect them and how to resolve them.

Maximo — How to detect database connection leak
Maximo — How to solve database connection leak

Here is the crux –

To free up locked database connect in maximo, IBM has introduce 3 new system properties (7.5.0.3 onwards) –

mxe.db.closelongrunconn – default is false.

mxe.db.longruntimelimit – default 180 minutes.

mxe.db.detectlongrunconninterval – default is 30 minutes. This is the frequency in which the long running connections are checked. Cannot be less then 30 minutes.

The mxe.db.closelongrunconn property when set to true will close the lost connections if the connections have been held for greater than mxe.db.longruntimelimit time and were not used by any processes in that time duration.


To detect database connection leak, turn Maximo dbconnection watchdog logger to INFO and collect the logs for 1-2 days. If there are any connection leaks the logs will show some thing like this –

[INFO] BMXAA7084I – The DbConnectionWatchDog class has been trying to close the database connection for: 230233001 ms

DbConnectionWatchDog:Db Connection reference id=436107 SPID=397
Create time:1302986384636
Life time:230233001 ms

The Logger indicates that the connection as being held for 230233001 ms i.e. approximately 64 hours. By looking in the logs at approximately 64 hours back, one should be able to find the stack trace of where this connection was established

Disabling The Warning Message Shown While Closing The Browser

Sometimes when I am in a hurry and I have to close my browser which has multiple Maximo sessions open, it is very frustrating to see each tab asking me to click on ‘Leave This Page’ to close it even after signing out from one of the tabs.

Close Confirmation

There is a property in Maximo – webclient.exitwarn to disable this. By default its set to 1. Setting it to 0 from the system property application will disable the confirmation box. This doesn’t require a restart to take effect as this property is Live Refresh enabled.