ETL Process time efficiency Decreased | SQL Server Performance Forums
SQL Server Performance Forum – Threads Archive
ETL Process time efficiency DecreasedHi Team,
We have an ETL process running in which,around 3 million records will be Transformed and loaded in to respective tables for BO purpose.A week back,this process was consuming 3 hours to complete the task.But since a week,the time frame has increase to 5 hours.
I am really not able to find the bottleneck !
I would like to know the possibilities of improving this issue. Looking forward,
The Articles section of this site is a good starting point – http://www.sql-server-performance.com/articles/per/index.aspx
I heartily recommend the articles on performance audit, starting with this one – http://www.sql-server-performance.com/articles/per/performance_audit_p1.aspx – work through all parts. As a bonus, a good understanding of the issues will help you become a better programmer/developer.
Welcome to the forums.
As you say it is a huge process to process the millions of records it would help if you can also specify the underlying hardware, processes that are running when this job is executed. Also the database maintenance practices on thsi database. THe underlying disk would help to speed up the performance to some extent and also the TSQL used in this import process. So if you can shell out these details we can help out more in detail.