Home All Groups Group Topic Archive Search About

Programming SSIS - AttachPathAndPropagateNotifications problem

Author
12 Nov 2005 6:46 AM
Slawomir Zurek
Hello,

I'm trying to build SSIS package from my application, and encountered a big
performance problem - AttachPathAndPropagateNotifications method semm very
slowly when Dataflow contains some more components.
Below description of my scenario:

- i'm building SSIS for transform facts table to my data warehouse; its
single table with about 50 fields; 40 of them are relation fileds to
dimensions
- dimensions are imported earlier, and every dimension  has own surrogate
key
- each relation field value must be change into proper surrogate key value
or -1 if none found - so for every dimension i create something like this:

                |
                Lookup to dimension
                 |     \
                 |    derived column for -1 value
                 |    /
                Union

so my fact dataflow contains about 40 constructions like this.

Now when i generate this package i encountered that after creating lookups
for 20 dimension AttachPathAndPropagateNotifications method slows down very
much - it takes about 5-10 seconds for dimensions  21-24 and this time grows
very much for the following dimensions (after 24th dimension it takes about
2 minutes to create described  fragment - 3 components, and four paths

Is there any way to speed up creating such a dataflow ?
Or have i change whole algorithm of my dataflow (how then should i build
data flows with many lookups/components) ?

Thanks in advance
    Slawomir Zurek
    Comarch S.A
    Poland

PS: sorry for my english, but i hope you could catch the sense....

AddThis Social Bookmark Button