I've been seeing some strange requests from developers these days. Here are few examples.
"I have a relational source and I want to take data for a column whose value is like '%XYZ%'. I want to accomplish this without SQL override on Source Qualifier".
"I have to create a copy of my target file in the archive location. Shall I create two target object instances in the mapping to acheive this? I can't use post-session command task".
...
...
...
It is certainly a surpirse to know as to why they want to avoid the obvious choice. All these developers have complained about development rules and guidelines imposing these restrictions at their organization level.
I have a question to the organizations.
"If you don't want to utilize the best options provided by PowerCenter to make your development/maintenance life easier, why do you opt to use PowerCenter at all?"
With the newer versions of PowerCenter, there is lot of flexibility provided to the developer to make his life lot easier than what it used to be during PowerMart times.
Imagine you code a mapping for the first developer question about '%XYZ%' and used a filter instead of overriding the SQL. If your soure table has 100 billion records and only 10 out of them satisfy '%XYZ%' condition. This will be the worst nightmare in support perspective. These are the mappings that eat into the organisations support budget and maintenance budget.
Why restrict the developer and suffer project financial and quality hammering?
There is another big problem with this kin of corporate behaviour. The inherent innovation capabilities of the developer are supressed and there is every possibility that the developer fails to use the best option when he/she moves to other organizations which give full freedom to the developers.
If this is the situation of the experienced developers, imagine some one who begin ETL PowerCenter development in these kind of restrictive-organiations. These poor developers would not even know that a certain option can be used in PowerCenter.
I hope this articles warns the organizations of a possible danger!
Tuesday, December 30, 2008
Subscribe to:
Post Comments (Atom)
Posts
5 comments:
hey, we live in the world in which multitude of factors, some obvious and some may not, influence our decisions ( u owe me a can of bear for this gyan).
And regarding practice in consideration, I think you already know the reason, albeit subconsciously: "...The inherent innovation capabilities of the developer are supressed and there is every possibility that the developer fails to use the best option...".
In today's client-server, oh! vendor, architecture, performance of code takes backseat to its so-called maintenance.
Unfortunately, there is a reason for this. Clients worry that if a developer writes performance-superior code, either using best technical features or better and not-so-obvious algorithm, it will become complicated -- i really wonder how they measure complicity!. And, once the developer leaves, who will manage the code (oh! my gosh, I'll doom).
This thinking is not only restricted to client. Even our so-called project managers fear that innovative developer creates dependency in a team, because of not-so-easily-manageable code, and will ask for better appraisal and higher perks (u know, these PMs always and only worry about appraisal).
So, don't worry!!!
Regards,
Pushkar
I really have to disagree with you. I know how PMs work and projects run. There are all kinds of people in the society and they have their own restrictions.
If overriding an SQL in Source Qualifier or using command tasks is going to create a fear in the hearts of the PM, then it is a serious issue to be addressed at the next level.
If you are really writing superior code, then you have every right to be appraised.
If you are afraid of maintaining an efficient code after a developer leaves, then suffer the losses. This is what I had to say to the so-called PMs and "client-server" guys.
Cheers
Sarma.
i am totally with radha krishna on this.
Hi Radhakrishna,
This Is Srikanth.
FIrst of all let me congratulate for your blog. Its really good and never seen one like this for Informatica.
When it comes to the fact that, organisations do not allow developers to use query override (source /lookup), I see a reason behind it.
That is: Data lineage.
If we override most of the logic then, its very difficult to do the impact analysis. Presently, I am part of a ware house which is been in place for more then 2 decades. ETL is accomplished by mainframe, Unix, Sql and also Informatica.
There are umpteen no of jobs, in my present project which perform most of the logic in the query overrides by joining multiple tables.
If we have to enhance the mapping/ table, then its a night mare for developers , since we have to check the entire query overrides.
The point I am trying to drive home is:
I agree that We need to push the processing to database but not at the cost of sacrificing data lineage. One best option I could think of is Pushdown optimisation.
smiles,
Srikanth
BASEL - The Adaptive Partner Program focuses on enhancing partnerships in order to successfully sell, deliver and support the Adaptive suite of products on a global scale. A key requirement for an Adaptive partner is that they are able to sell and deliver the product, as well as provide first line support to their clients.
Post a Comment