|
database
newsgroups
|
|||||||||||||||||||||||
|
|||||||||||||||||||||||
Programming SSIS - AttachPathAndPropagateNotifications problemI'm trying to build SSIS package from my application, and encountered a big performance problem - AttachPathAndPropagateNotifications method semm very slowly when Dataflow contains some more components. Below description of my scenario: - i'm building SSIS for transform facts table to my data warehouse; its single table with about 50 fields; 40 of them are relation fileds to dimensions - dimensions are imported earlier, and every dimension has own surrogate key - each relation field value must be change into proper surrogate key value or -1 if none found - so for every dimension i create something like this: | Lookup to dimension | \ Union| derived column for -1 value | / so my fact dataflow contains about 40 constructions like this. Now when i generate this package i encountered that after creating lookups for 20 dimension AttachPathAndPropagateNotifications method slows down very much - it takes about 5-10 seconds for dimensions 21-24 and this time grows very much for the following dimensions (after 24th dimension it takes about 2 minutes to create described fragment - 3 components, and four paths Is there any way to speed up creating such a dataflow ? Or have i change whole algorithm of my dataflow (how then should i build data flows with many lookups/components) ? Thanks in advance Slawomir Zurek Comarch S.A Poland PS: sorry for my english, but i hope you could catch the sense.... |
|||||||||||||||||||||||