Duke Consulting Tips & Tricks











The starting point for performance tuning is to get and track row counts. Rows per second is the leading indicator of problems. If rows start slowing down over time then why. EtlStats.zip conatins jobs and documentation to get row counts. We run this job at the end of each job sequence. The results are stored in Oracle. The DDL to create the tables is included. Also included is DSaveAsBmp.bat which will create bmp images for each job. Makes for great documentation.

Soon I will include a batch job to document all jobs and routines in your project. This is based on JobReport.dsx available on ADN.

More jobs and tips coming soon.

If you have any problems then email me.

Information: Kim_G_Duke@hotmail.com


Description Link
Documentation for EtlStats jobs below.
Documentation generated by KgdGenHtml jobs. 		EtlStats_Index
Ds7.1 Jobs get row counts on all jobs in a sequence, all jobs or just one job. Now with missing files. EtlStats.zip
Installing EtlStats. Install Video
Sample reports from EtlStats. Sample records for ETL_QA tables. EtlStatsReports.zip
Automated QA. Setting up QA Completeness Reports. QA Completeness Setup
Jobs and routines to extract information about jobs. DsRepository.zip
Documentation for KgdGenHtml jobs below.
Documentation generated by these jobs. 		KgdGenHtml_Index
These jobs will genrate Html documentation for all jobs
and routines in a project.
Uses JobReport routine downloaded from ADN.
Will display bmp images created by DSaveAsBmp.bat. 		KgdGenHtml.zip
Documentation for routines below.
Documentation generated by KgdGenHtml jobs. 		KgdRoutine_Index
Misc routines.
1. KgdCreateDdlMetadata - Generate create table scripts from DS_METADATA.
2. KgdDebugMetadata - Display metadata mismatches.
3. KgdFixLongDesc - Put a standard long description on a job.
4. KgdFixLongDescRtn - Put a standard long description on a routine.
5. KgdGetLastWarning - Get last warning message from log file.
6. KgdGetShell - Get shell based on OS (DOS or SH). (Thanks Ray)
7. KgdGetSlash - Get slash based on OS. (Thanks Ray)
8. KgdMakeDir - Create a all directories in path like \KimD\Misc\Whatever.
9. KgdParseSeqFile - Compare first row of sequential file with column names to lengths.
10. KgdRemoveDir - Remove a all directories in path like \KimD\Misc\Whatever.
 		KgdRoutines.zip
DataStageBackup.bat will backup all DataStage projects on a server.
DSaveAsBmp.bat will create bmp images for all jobs in a project.
Keep clicking on OK button till done. 		Scripts.zip
My help files. Very crude text files.
Korn Shell, Perl, Universe and Vi help files. 		Help.zip
Xml Best Practices
 		XmlPackBestPractices.zip
Tech Tips From Ascential now IBM
 		TwoTechTips.zip