|
Collapse All | Expand All | Show Default |
| DSJobReportDb
DataStage Job Report Generated 2005-02-15 08:26:49pm |
| - Job Design |
|
|---|
| - Job Information |
| - Job Properties (7) |
| Property | Value |
| Object Type | Server |
| Full Description | Job: DSJobReportDb
================================================================= Description: This job will get row counts for all jobs in a sequence. Uses dsjob -report in a before routine. Originally it was based on DSJobReport downloaded from ADN. There are lots of changes to the original. 1. All XML is validated. 2. The XML had to be split into 3 parts. a. Because JobStats does not have any repeatable fields. b. Because RowStats and ParamStats are in different repeatable groups. 3. Transformer added for RowStats and ParamStats to validate that there was data a. XML plugin gave square root of negative number errors otherwise b. XML also gave divide by zero errors. c. XML errors crashed the job 4. dsjob -report does as a before routine instead of filter a. Because filter was unstable crashed too much b. store XML history in files like jobName.xml 5. Added driver job a. To run as last job in sequence. b. To run all jobs c. To run one job d. To trap multiple instance jobs (need to run as job.invocationid) The hash files store the last run values. ================================================================= Parameters: 1. TargetDSN 2. TargetPwd 3. TargetUser 4. dsjobcmd - dsjob -report 5. jobName 6. projName - @WHO ================================================================= Sources: dsjobcmd = dsjob -report XML ================================================================= Targets: 1. JobStatusLastRun 2. ETL_JOB_HIST 3. JobRowsLastRun 4. ETL_ROW_HIST 5. JobParamLastRun 6. ETL_PARAM_HIST ================================================================= To use in next job: DSJobReportDbDriver - parent of this SeqJobStatus - run before this job a. ExtractJobStatusSeq - creates ./Staging/JobStatusStg.txt b. BuildJobStatusHash - used as lookup in this job ================================================================= Modification History ================================================================= Developer Date Modification description ======== ========= ============================================== kduke 2004-06-03 created ======== ========= ============================================== |
| Category | EtlStats |
| Multiple Instance | True |
| Job Version Number | 40.1.5 |
| Hash File Cache Sharing | False |
| Before Routine | DSU.ExecSH(#dsjobcmd# #projName# #jobName# XML >KimD/#jobName#.xml) |
| - Parameters (10) |
| Parameter Name | Prompt | Default Value | Help Text |
| dsjobcmd | dsjob command | ../../DSEngine/bin/dsjob -report | the command line options for dsjob |
| projName | project name | DwNav | the name of the project |
| jobName | job name | DSJobReportDb.1 | the name of the job (or its id) |
| TargetDSN | TargetDSN | kduke | |
| TargetUser | TargetUser | etluser | |
| TargetPwd | TargetPwd | LDH@9:VA=93L0O5I<;J| |
|
| SourceDSN | SourceDSN | kduke | not used |
| SourceUser | SourceUser | etluser | |
| SourcePwd | SourcePwd | LDH@9:VA=93L0O5I<;J| |
|
| FilePath | FilePath | ./ |
| - Stages (18) |
| - Source Stages (4) |
| DSJobReport2 (sequential file) |
| JobStatusHash (hash file) |
| DSJobReport1 (sequential file) |
| DSJobReport3 (sequential file) |
| - Processing Stages (8) |
| XfmXml2 (xmlinput) |
| XfmStats (transformer) |
| CheckForRows (transformer) |
| CheckForParams (transformer) |
| XfmRows (transformer) |
| XfmXml1 (xmlinput) |
| XfmXml3 (xmlinput) |
| XfmParams (transformer) |
| - Target Stages (6) |
| JobParamLastRun (hash file) |
| ETL_JOB_HIST (oraoci8) |
| ETL_PARAM_HIST (oraoci8) |
| JobStatusLastRun (hash file) |
| ETL_ROW_HIST (oraoci8) |
| JobRowsLastRun (hash file) |
| - Source Stages (4) |
| - DSJobReport2 (sequential file) |
| - JobStatusHash (hash file) |
| - Stage Properties (2) |
| Property | Value |
| SQL Null Value | Standard DSEngine |
| UniVerse Stage Compatibility | False |
| - Output Links (1) |
| - DSJobReport1 (sequential file) |
| - DSJobReport3 (sequential file) |
| - Processing Stages (8) |
| - XfmXml2 (xmlinput) |
| - Stage Properties (12) |
| Property | Value |
| validate_input | 1 |
| log_reject_errors | 0 |
| xslt_warning_mapping | DS_WARNING |
| xslt_error_mapping | DS_REJECT |
| xslt_fatal_mapping | DS_REJECT |
| accept_nulls | 0 |
| replace_nulls | 0 |
| discard_empty_values | 0 |
| include_namespaces | 0 |
| format_serialized_chunks | 0 |
| enable_grammar_caching | 0 |
| xml_validation_level | 0 |
| - Input Links (1) |
| - Output Links (1) |
| - XfmStats (transformer) |
| - CheckForRows (transformer) |
| + Stage Variables (0) |
| - Input Links (1) |
| - Output Links (1) |
| + RowsOut triggers XfmXml2 when Index (xml_report2.XMLDOC, 'RowCount', 1) > 0 |
| - CheckForParams (transformer) |
| + Stage Variables (0) |
| - Input Links (1) |
| - Output Links (1) |
| + ParamsOut triggers XfmXml3 when Index (xml_report3.XMLDOC, 'ParamSet', 1) > 0 |
| - XfmRows (transformer) |
| - XfmXml1 (xmlinput) |
| - Stage Properties (12) |
| Property | Value |
| validate_input | 1 |
| log_reject_errors | 0 |
| xslt_warning_mapping | DS_WARNING |
| xslt_error_mapping | DS_REJECT |
| xslt_fatal_mapping | DS_REJECT |
| accept_nulls | 1 |
| replace_nulls | 0 |
| discard_empty_values | 0 |
| include_namespaces | 0 |
| format_serialized_chunks | 0 |
| enable_grammar_caching | 0 |
| xml_validation_level | 0 |
| - Input Links (1) |
| - Output Links (1) |
| - XfmXml3 (xmlinput) |
| - Stage Properties (12) |
| Property | Value |
| validate_input | 1 |
| log_reject_errors | 0 |
| xslt_warning_mapping | DS_WARNING |
| xslt_error_mapping | DS_REJECT |
| xslt_fatal_mapping | DS_REJECT |
| accept_nulls | 0 |
| replace_nulls | 0 |
| discard_empty_values | 0 |
| include_namespaces | 0 |
| format_serialized_chunks | 0 |
| enable_grammar_caching | 0 |
| xml_validation_level | 0 |
| - Input Links (1) |
| - Output Links (1) |
| - XfmParams (transformer) |
| - Target Stages (6) |
| - JobParamLastRun (hash file) |
| - Stage Properties (2) |
| Property | Value |
| SQL Null Value | Standard DSEngine |
| UniVerse Stage Compatibility | False |
| - Input Links (1) |
| - ETL_JOB_HIST (oraoci8) |
| - Stage Properties (4) |
| Property | Value |
| DATABASE | #TargetDSN# |
| USERID | #TargetUser# |
| PASSWORD | HD:@I:V@L93?0GEI<;J<0K1GB |
| ISOPS | NO |
| - Input Links (1) |
| - ETL_PARAM_HIST (oraoci8) |
| - Stage Properties (4) |
| Property | Value |
| DATABASE | #TargetDSN# |
| USERID | #TargetUser# |
| PASSWORD | HD:@I:V@L93?0GEI<;J<0K1GB |
| ISOPS | NO |
| - Input Links (1) |
| - JobStatusLastRun (hash file) |
| - Stage Properties (2) |
| Property | Value |
| SQL Null Value | Standard DSEngine |
| UniVerse Stage Compatibility | False |
| - Input Links (1) |
| - ETL_ROW_HIST (oraoci8) |
| - Stage Properties (4) |
| Property | Value |
| DATABASE | #TargetDSN# |
| USERID | #TargetUser# |
| PASSWORD | HD:@I:V@L93?0GEI<;J<0K1GB |
| ISOPS | NO |
| - Input Links (1) |
| - JobRowsLastRun (hash file) |
| - Stage Properties (2) |
| Property | Value |
| SQL Null Value | Standard DSEngine |
| UniVerse Stage Compatibility | False |
| - Input Links (1) |
