Clean up after your batch jobs!

There is now standard functionality in Dynamics 365 for Finance and Supply Chain to clean up Batch Job History entries created from batch jobs. In AX 2012, you had to write something custom to handle this. There are two different jobs available to help clean up your Batch History table. To see your Batch Job history, go to System Administration > Inquiries > Batch Jobs, switch to the enhanced view, then click on Batch Job History. This will show you the history for the specific Batch Job you were viewing. Also, as a hacky way to get there, you can use a special URL specifying the Menu Item manually, like so: https://usnconeboxax1aos.cloud.onebox.dynamics.com/?mi=BatchJobHistory. This will show all entries for all Batch Jobs. However, this may be removed with any release so it may not always be there. It appears Microsoft is trying to remove this form from menus and instead have it be visible only from a selected Batch Job. This table contains all of the history for a Batch Job depending on how a batch job is setup. Typically, to prevent you Batch Job history table from collecting too much stuff, you can set the option "Save Job to History" to Errors Only on the batch job so you only see when an issues occurs. This may not be an option for every batch job. For instance, if you invoice in batch, you may want to review the result of an invoice batch as an individual invoice will fail but the batch will post as successful. Regardless, records are going to build in this table and its be a good idea to keep it pruned so you can find your actionable items quickly should you need to. There are 2 options available now.

Batch Job History Clean-up

The first option can be found in System Administration > Periodic Tasks > Batch Job History Clean-up. This is a simple dialog that asks for a limit in days. Any records older than the specified limit will be deleted regardless of status or company.

Although this looks simple, behind the scenes, it has some pretty interesting stuff. First, If you are deleting more than 500k records and the percentage of the table you are deleting is equal to or greater than 75%, it takes a different execution path to a stored procedure and doesn't use standard X++ transactions. So if you have a lot of accumulated records, you need not worry. Microsoft planned for this as the stored procedure is much more performant as a set based operation rather than a row based operation. The proc is SysTruncateBatchHistory if you'd like to check it out. It's also hooked into some instrumentation that would surface in LCS so you can keep an eye on what is going on via LCS rather than rely on the batch job reporting, if you like. Regardless of the code path it takes, it does the same basic things. They all will identify what records in the Batch Jobs History table (BatchJobHistory) based on their age, delete them, then cleanup records in Batch Tasks History (BatchHistory) and Has Conditions (BatchConstraintHistory) tables related to the BatchJobHistory records that were deleted. The stored proc uses a TRUNCATE so that's how it is able to be much faster for large clean-up operations.

Batch Job History Clean-up (Custom)

The section option can be found in System Administration > Periodic Tasks > Batch Job History Clean-up (Custom). This is similar to the first option but lets you get more granular with your options. This will let you specify that you only want to delete records that are older than 30 days, in then status ended for company USMF, like so:

If you don't specify any parameters besides days, this will revert back to using option 1 as it has a chance of being a lot more performant. This has no limit on what will be deleted so if you select a recort set that is 100 million records, this job will attempt to process them which may cause some locking issues, potentially halting all batch batch jobs that write to the Batch Jobs History table.. This option should be used for targeted engagements only. A good strategy is to use option 1 on a recurring schedule to clean up entries you will never need after a certain amount of time, like 180 days, and use option 2 to clean up records in a targeted fashion on a much smaller time horizon, like 30 days, plus some additional identifiers to narrow the scope of what is removed. This job operates similar to the last one but in a reverse order.  This will loop through all Batch Jobs History table (BatchJobHistory) records, first delete Has Conditions (BatchConstraintHistory) records for the given BatchJobHistory record, then delete all Batch Tasks History (BatchHistory) for the given BatchJobHistory then delete Batch Jobs History table (BatchJobHistory) all based on the query parameters.