注册 登录  
 加关注
   显示下一条  |  关闭
温馨提示!由于新浪微博认证机制调整,您的新浪微博帐号绑定已过期,请重新绑定!立即重新绑定新浪微博》  |  关闭

zjcjack的博客

 
 
 

日志

 
 

一次sqlldr性能测试案例  

2012-02-21 14:24:37|  分类: oracle |  标签: |举报 |字号 订阅

  下载LOFTER 我的照片书  |
$ sqlldr

SQL*Loader: Release 10.2.0.1.0 - Production on Tue May 19 16:06:21 2009

Copyright (c) 1982, 2005, Oracle. All rights reserved.


Usage: SQLLDR keyword=value [,keyword=value,...]

Valid Keywords:

    userid -- ORACLE username/password           
   control -- control file name                  
       log -- log file name                      
       bad -- bad file name                      
      data -- data file name                     
   discard -- discard file name                  
discardmax -- number of discards to allow          (Default all)
      skip -- number of logical records to skip    (Default 0)
      load -- number of logical records to load    (Default all)
    errors -- number of errors to allow            (Default 50)
      rows -- number of rows in conventional path bind array or between direct path data saves
               (Default: Conventional path 64, Direct path all)
bindsize -- size of conventional path bind array in bytes (Default 256000)
    silent -- suppress messages during run (header,feedback,errors,discards,partitions)
    direct -- use direct path                      (Default FALSE)
   parfile -- parameter file: name of file that contains parameter specifications
parallel -- do parallel load                     (Default FALSE)
      file -- file to allocate extents from      
skip_unusable_indexes -- disallow/allow unusable indexes or index partitions (Default FALSE)
skip_index_maintenance -- do not maintain indexes, mark affected indexes as unusable (Default FALSE)
commit_discontinued -- commit loaded rows when load is discontinued (Default FALSE)
readsize -- size of read buffer                  (Default 1048576)
external_table -- use external table for load; NOT_USED, GENERATE_ONLY, EXECUTE (Default NOT_USED)
columnarrayrows -- number of rows for direct path column array (Default 5000)
streamsize -- size of direct path stream buffer in bytes (Default 256000)
multithreading -- use multithreading in direct path 
resumable -- enable or disable resumable for current session (Default FALSE)
resumable_name -- text string to help identify resumable statement
resumable_timeout -- wait time (in seconds) for RESUMABLE (Default 7200)
date_cache -- size (in entries) of date conversion cache (Default 1000)

PLEASE NOTE: Command-line parameters may be specified either by
position or by keywords. An example of the former case is 'sqlldr
scott/tiger foo'; an example of the latter is 'sqlldr control=foo
userid=scott/tiger'. One may specify parameters by position before
but not after parameters specified by keywords. For example,
'sqlldr scott/tiger control=foo logfile=log' is allowed, but
'sqlldr scott/tiger control=foo log' is not, even though the
position of the parameter 'log' is correct.

0) 控制文件

install.ctl(含序列,不能设direct=true):

load data

infile '/orahome/wangnc/INSTALLDETAIL.dat'

append

into table sys_login_records_detail_1

fields terminated by ','

(

LOGIN_DATE DATE "yyyy-mm-dd" TERMINATED BY whitespace,

FILLER_1 FILLER,

GAME_ID,

FILLER_2 FILLER,

IP,

MAC,

NETBAR_ID,

PROVINCE,

CITY,

LOGIN_TIMES,

id POSITION(1:1) "sys_login_records_detail_seq_1.nextval"

)

install4.ctl(不含序列,可设direct=true):

Load data

infile '/orahome/wangnc/INSTALLDETAIL.dat'

append

into table sys_login_records_detail_1

fields terminated by ','

(

LOGIN_DATE DATE "yyyy-mm-dd" TERMINATED BY whitespace,

FILLER_1 FILLER,

GAME_ID,

FILLER_2 FILLER,

IP,

MAC,

NETBAR_ID,

PROVINCE,

CITY,

LOGIN_TIMES

)

1) 常规导入

sqlldr userid='emoa_sqlldr/"xxx"@db_esuite' \

control=/orahome/wangnc/install.ctl \

log=/orahome/wangnc/install1.log \

bad=/orahome/wangnc/install1.bad

##Space allocated for bind array:                  132352 bytes(64 rows)

##Elapsed time was:      00:01:45.85


2) 1000条提交一次

sqlldr userid='emoa_sqlldr/"xxx"@db_esuite' \

control=/orahome/wangnc/install.ctl \

log=/orahome/wangnc/install2.log \

bad=/orahome/wangnc/install2.bad \

rows=1000 readsize=2068000 bindsize=2068000

##Space allocated for bind array:                 2068000 bytes(1000 rows)

##Elapsed time was:      00:00:22.80

3) 10000条提交一次

sqlldr userid='emoa_sqlldr/"xxx"@db_esuite' \

control=/orahome/wangnc/install.ctl \

log=/orahome/wangnc/install3.log \

bad=/orahome/wangnc/install3.bad \

rows=10000 readsize=20680000 bindsize=20680000

##Space allocated for bind array:                20680000 bytes(10000 rows)

##Elapsed time was:      00:00:20.25

4) 设置direct=true, 含序列

sqlldr userid='emoa_sqlldr/"xxx"@db_esuite' \

control=/orahome/wangnc/install.ctl \

log=/orahome/wangnc/install4.log \

bad=/orahome/wangnc/install4.bad \

direct=true

##ORA-01400: cannot insert NULL into ("EMOA"."sys_login_records_detail_1"."ID")

5) 设置direct=true, 去掉序列字段最快的方式

#SQL> alter table emoa.sys_login_records_detail_1 drop column id;

sqlldr userid='emoa_sqlldr/"xxx"@db_esuite' \

control=/orahome/wangnc/install4.ctl \

log=/orahome/wangnc/install4.log \

bad=/orahome/wangnc/install4.bad \

direct=true

##Elapsed time was:      00:00:10.98

##但是如果有重复数据会把唯一索引置为unusable, 要确保数据源已剔重

6) 设置readsizebindsize到最大值高效又稳妥的方式最终采纳方案

sqlldr userid='emoa_sqlldr/"xxx"@db_esuite' \

control=/orahome/wangnc/install4.ctl \

log=/orahome/wangnc/install6.log \

bad=/orahome/wangnc/install6.bad \

rows=100160 readsize=20971520 bindsize=20971520 PARALLEL=TRUE

##Space allocated for bind array:                20970240 bytes(10160 rows)

##Elapsed time was:      00:00:14.36


--End--
  评论这张
 
阅读(132)| 评论(0)
推荐 转载

历史上的今天

评论

<#--最新日志,群博日志--> <#--推荐日志--> <#--引用记录--> <#--博主推荐--> <#--随机阅读--> <#--首页推荐--> <#--历史上的今天--> <#--被推荐日志--> <#--上一篇,下一篇--> <#-- 热度 --> <#-- 网易新闻广告 --> <#--右边模块结构--> <#--评论模块结构--> <#--引用模块结构--> <#--博主发起的投票-->
 
 
 
 
 
 
 
 
 
 
 
 
 
 

页脚

网易公司版权所有 ©1997-2017