0
4.5GB mysqldump 파일의 데이터베이스를 AWS Aurora DB 인스턴스로 가져 오려고합니다. 내 DB에는 ~ 80 개의 테이블이 있으며 최대 테이블에는 1,300 만 개의 행이 있습니다 (나머지는 훨씬 작음). 내 덤프 파일에는 에 지정된 max_allowed_packet에 따라 mysqldump이 각각 64MB로 제한되는 다중 값 삽입이 있습니다.AWS 오로라 대량 가져 오기 오류가 발생했습니다.
나는 가져 오기를 초기에 여러 가지 문제가 있었다, 그러나 나는 오로라 파라미터 그룹 옵션
의 max_allowed_packet = 1073741824 (1GB)
아웃 (wait_timeout) = 10,800 (3 시간), 즉
를 설정하여 그 문제를 해결 할 수 있었다 net_read_timeout = 10800
net_write_timeout = 10800
interactive_timeout = 10800
처음에는 오로라 인스턴스가 db.t2.small (2GB RAM) 이었지만 가져 오기를 시도했을 때 mysql -u ... mydb < dump.sql EC2 인스턴스 (m3.medium, 4GB RAM)에서 프로세스가 실패한 후 1 분 동안 실행되지 않았습니다. RDS 로그는 메모리 부족 오류라고 알려줍니다. 오로라 인스턴스를 db.t2.medium (4GB RAM)까지 올렸지 만 같은 종류의 메모리 부족으로 약 20 분이 지나면 프로세스가 다시 실패했습니다.
다음 인스턴스 유형 (15GB RAM)으로 이동하고 싶지는 않지만 어쨌든 필자는이를 이해하지 못합니다. 내가 사용하고있는 m3.medium EC2 인스턴스의 로컬 MySQL 서버에 동일한 mysqldump 파일을 정기적으로 가져 왔으며 아무런 문제가 없었습니다. 수입까지 ~ 40 분이 소요됩니다.
가 여기에 내가 노력 마지막으로 가져온에서 오로라 오류 로그입니다 :
Available memory is low. Trying to avoid OOM crash: system KB: 4050724 available KB: 101748 low-threshold KB: 202536 decline query: no tune caches: no kill query: no kill connection: no
OOM crash avoidance result: success: no num success: 0 system KB: 4050724 available KB: 111688 low-threshold KB: 202536 recovery time: 11 num declined query: 0 num killed query: 0 num killed connection: 0
OOM crash avoidance result: success: yes num success: 1 system KB: 4050724 available KB: 529464 low-threshold KB: 202536 recovery time: 24 num declined query: 0 num killed query: 0 num killed connection: 0
Available memory is low. Trying to avoid OOM crash: system KB: 4050724 available KB: 200956 low-threshold KB: 202536 decline query: no tune caches: no kill query: no kill connection: no
OOM crash avoidance result: success: yes num success: 2 system KB: 4050724 available KB: 556020 low-threshold KB: 202536 recovery time: 5 num declined query: 0 num killed query: 0 num killed connection: 0
Available memory is low. Trying to avoid OOM crash: system KB: 4050724 available KB: 170392 low-threshold KB: 202536 decline query: no tune caches: no kill query: no kill connection: no
OOM crash avoidance result: success: yes num success: 3 system KB: 4050724 available KB: 554108 low-threshold KB: 202536 recovery time: 7 num declined query: 0 num killed query: 0 num killed connection: 0
Available memory is low. Trying to avoid OOM crash: system KB: 4050724 available KB: 194900 low-threshold KB: 202536 decline query: no tune caches: no kill query: no kill connection: no
OOM crash avoidance result: success: yes num success: 4 system KB: 4050724 available KB: 554340 low-threshold KB: 202536 recovery time: 8 num declined query: 0 num killed query: 0 num killed connection: 0
Available memory is low. Trying to avoid OOM crash: system KB: 4050724 available KB: 198780 low-threshold KB: 202536 decline query: no tune caches: no kill query: no kill connection: no
OOM crash avoidance result: success: no num success: 4 system KB: 4050724 available KB: 133160 low-threshold KB: 202536 recovery time: 11 num declined query: 0 num killed query: 0 num killed connection: 0
OOM crash avoidance result: success: yes num success: 5 system KB: 4050724 available KB: 556540 low-threshold KB: 202536 recovery time: 25 num declined query: 0 num killed query: 0 num killed connection: 0
Available memory is low. Trying to avoid OOM crash: system KB: 4050724 available KB: 170224 low-threshold KB: 202536 decline query: no tune caches: no kill query: no kill connection: no
OOM crash avoidance result: success: yes num success: 6 system KB: 4050724 available KB: 579368 low-threshold KB: 202536 recovery time: 1 num declined query: 0 num killed query: 0 num killed connection: 0
Available memory is low. Trying to avoid OOM crash: system KB: 4050724 available KB: 175612 low-threshold KB: 202536 decline query: no tune caches: no kill query: no kill connection: no
<jemalloc>: Error in mmap(): err: 12, msg: Cannot allocate memory
<jemalloc>: Error in malloc(): out of memory
<jemalloc>: System-wide: MemTotal: 4050724kb, MemFree: 137440kb, Buffers: 20428kb, Cached: 62340kb, Active: 2188968kb, Dirty: 204kb, Inactive: 37228kb, Mapped: 41592kb
<jemalloc>: terminating process due to out of resources
10:58:03 UTC - mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
key_buffer_size=16777216
read_buffer_size=262144
max_used_connections=6
max_threads=90
thread_count=5
connection_count=5
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 63814 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x2ab59d623000
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 2ab524642c08 thread_stack 0x40000
/rdsdbbin/oscar/bin/mysqld(my_print_stacktrace+0x2c)[0x9897ec]
/rdsdbbin/oscar/bin/mysqld(handle_fatal_signal+0x491)[0x6f0651]
/lib64/libpthread.so.0(+0xf5b0)[0x2ab5165405b0]
/lib64/libc.so.6(gsignal+0x39)[0x2ab51925cbe9]
/lib64/libc.so.6(abort+0x148)[0x2ab51925dfe8]
/rdsdbbin/oscar/lib/libjemalloc.so(malloc+0x1226)[0x2ab5160f62e6]
/rdsdbbin/oscar/bin/mysqld(my_malloc+0x25)[0x986525]
/rdsdbbin/oscar/bin/mysqld(alloc_root+0x8f)[0x9824bf]
/rdsdbbin/oscar/bin/mysqld[0x5b57a5]
/rdsdbbin/oscar/bin/mysqld(_Z10MYSQLparsePv+0xc1ce)[0x826ade]
/rdsdbbin/oscar/bin/mysqld(_Z9parse_sqlP3THDP12Parser_stateP19Object_creation_ctx+0xb5)[0x772175]
/rdsdbbin/oscar/bin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0xf2)[0x772502]
/rdsdbbin/oscar/bin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xf43)[0x774003]
/rdsdbbin/oscar/bin/mysqld(_ZN22OscarSchedulerConsumer7consumeEjj+0xd3)[0x803963]
/rdsdbbin/oscar/bin/mysqld(_ZN22OscarSchedulerConsumer5startEv+0x98)[0x803a98]
/rdsdbbin/oscar/bin/mysqld(_ZN22OscarSchedulerConsumer11drain_queueEPv+0x6a)[0x803cda]
/lib64/libpthread.so.0(+0x7f18)[0x2ab516538f18]
/lib64/libc.so.6(clone+0x6d)[0x2ab51930bb2d]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (2ab5ad400010): INSERT INTO `Documents` VALUES (478555572,150317,1321817,1,9,627609600,0,60,5471267,0,639014400,''),(478555571,150317,1321816,1,1,623980800,0,60,0,0,623980800,''),(478555575,150318,1321820,1,1,623980800,0,60,0,0,623980800,'')
Connection ID (thread ID): 6
Status: NOT_KILLED
binlog_format 매개 변수를 설정하여 바이너리 로깅을 사용하지 않도록하고 인스턴스를 다시 시작 시도했습니다,하지만 난 쿼리
select @@binlog_format를 실행할 때 나는 결과를 얻을 수
명세서.