database load is slow
DBAsupport.com Forums - Powered by vBulletin
Results 1 to 6 of 6

Thread: database load is slow

  1. #1
    Join Date
    Aug 2001
    Location
    Manchester, UK
    Posts
    86
    Hi !

    I am trying to populate my main table from a dummy table. Dummy table contains 5 million records. I am performing some checks on the records and then populating them in the main table.

    Initially my log files tells me that 1000 records are processed every 20 seconds but over the time, after 1 million records the processing time goes upto 8-10 mins / 1000

    What can be the problem??

    Kind regards,
    -- Anurag.
    OCP Application Developer
    ---------------------------------------------------------
    "Be not afraid of growing slowly. Be afraid only of standing still."

  2. #2
    Join Date
    Jul 2001
    Location
    Slovenia
    Posts
    422
    Does 'performing some checks' include querying main table?
    Or do you have some indexes on main table being populated along?

  3. #3
    Join Date
    Aug 2001
    Location
    Manchester, UK
    Posts
    86
    Yes, Checking involves some queries on main table. Also indexes are created on main table.

    -- Anurag.
    OCP Application Developer
    ---------------------------------------------------------
    "Be not afraid of growing slowly. Be afraid only of standing still."

  4. #4
    Join Date
    Jul 2001
    Location
    Slovenia
    Posts
    422
    Look if you can remove some indexes and create them after load.
    Evaluate if you can somehow implement needed checks with processing sets of records, not row by row.

  5. #5
    Join Date
    Mar 2002
    Location
    Mesa, Arizona
    Posts
    1,204
    Can you load this data into a new table .. (ex. create table as .. select w/ NOLOGGING)?

    or does this data get appended to an existing table?
    "I do not fear computers. I fear the lack of them." Isaac Asimov
    Oracle Scirpts DBA's need

  6. #6
    Join Date
    Aug 2001
    Location
    Manchester, UK
    Posts
    86
    Well the scenerio is like this.

    We have a sales table and the client sends all his last month sales vochers to us. They capture the vouchers from different call centers. The feed is of around 5 million records.

    We need to enter only those vouchers which do not exist in sales table so check is there on 5 keys.

    Thanks for all your help.
    -- Anurag.
    OCP Application Developer
    ---------------------------------------------------------
    "Be not afraid of growing slowly. Be afraid only of standing still."

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width