{"id":1491,"date":"2020-02-11T23:18:40","date_gmt":"2020-02-11T14:18:40","guid":{"rendered":"https:\/\/oboki.net\/?p=1491"},"modified":"2020-02-11T23:18:50","modified_gmt":"2020-02-11T14:18:50","slug":"error-parsing-row-%ed%98%84%ec%83%81","status":"publish","type":"post","link":"https:\/\/oboki.net\/workspace\/data-engineering\/database\/impala\/error-parsing-row-%ed%98%84%ec%83%81\/","title":{"rendered":"[Impala] Error parsing row \ud604\uc0c1"},"content":{"rendered":"<p>impala \uc5d0\uc11c parquet \ud14c\uc774\ube14\uc744 \ubcf4\uc815\ud560 \uc77c\uc774 \uc788\uc5b4\uc11c, CTAS\ub85c \uc77c\ubd80\ubd84\ub9cc \ubc31\uc5c5\ud574\ub450\uace0 \ud639\uc2dc \ubb38\uc81c\uac00 \uc788\ub294\uc9c0 \ucffc\ub9ac\ub97c \ud574\ubd24\ub294\ub370 \ub2e4\uc74c\uacfc \uac19\uc740 \uc5d0\ub7ec\uac00 \ubc1c\uc0dd\ud588\ub2e4.<\/p>\n<pre><code class=\"language-log\">Error converting column: 126 to DOUBLE\nError parsing row: file: hdfs:\/\/cluster01\/user\/hive\/warehouse\/imp.db\/tab_dpcn_info_l\/a7422143a7852306-8b0cbcb600000005-1957752130_data.0, before offset: 629145600<\/code><\/pre>\n<p>\uc5d0\ub7ec \uba54\uc2dc\uc9c0 \uc790\uccb4\ub294 126\ubc88 \uceec\ub7fc\uc744 DOUBLE \ub85c \ud615\ubcc0\ud658 \ud558\ub294\ub370 \uc2e4\ud328\ud588\uace0 \ud574\ub2f9 \ubb38\uc81c\uac00 \ubc1c\uc0dd\ud55c \ub370\uc774\ud130\uac00 \uc704 <code>hdfs:\/\/..<\/code> \uacbd\ub85c\uc5d0, <code>offset<\/code> \ubc88\uc9f8 \uc904\uc5d0 \uc788\ub2e4\ub294 \ub0b4\uc6a9\uc778\ub370, CTAS\ub85c \ubcf5\uc81c\ud574\uc11c \ub9cc\ub4e0 \ud14c\uc774\ube14\uc744 csv \ud615\ud0dc\uc758 \ud14d\uc2a4\ud2b8 \ud30c\uc77c\ub85c \uc800\uc7a5\ud560 \ub54c\uc5d0\ub294 \uc704\uc640\uac19\uc740 \ubb38\uc81c\uac00 \ubc1c\uc0dd\ud588\uace0, \ub2e4\uc2dc parquet \ud3ec\ub9f7\uc73c\ub85c \uc800\uc7a5 (<code>stored as parquet<\/code>)\ud588\uc744 \ub54c\uc5d0\ub294 \uc5d0\ub7ec \ubc1c\uc0dd \uc5c6\uc774 \uc815\uc0c1 \ub3d9\uc791\ub418\uc5c8\ub2e4.<\/p>\n<p><strong>\uacb0\ub860\uc740 csv\ub294 \ubd88\uc644\uc804\ud55c \ud30c\uc77c \ud3ec\ub9f7\uc73c\ub85c\uc11c \uc0ac\uc6a9\uc2dc \uc8fc\uc758\ub97c \ud574\uc57c\ud558\uba70, parquet \ud3ec\ub9f7\uc744 \uc774\uc6a9\ud558\ub294 \uc2b5\uad00\uc744 \ub4e4\uc774\uc790.<\/strong><\/p>\n<p>\ubb38\uc81c \uc6d0\uc778\uc744 \ucc3e\ub2e4\uac00 \ud30c\uc77c\uc744 \uae4c\ubcf4\ub294 \uc218\uace0\ub85c\uc6c0\uc744 \uac10\uc218\ud55c \ub355\ubd84\uc5d0 \uc54c\uac8c\ub41c \uc0ac\uc2e4\uc740, CDH\ub294 csv \ud615\ud0dc\uc640 \uac19\uc740 \ud14d\uc2a4\ud2b8 \ud30c\uc77c\ub85c \ud14c\uc774\ube14\uc744 \uc800\uc7a5\ud560 \ub54c delimeter\ub85c <code>^A<\/code>\ub97c \uc0ac\uc6a9\ud55c\ub2e4.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>impala \uc5d0\uc11c parquet \ud14c\uc774\ube14\uc744 \ubcf4\uc815\ud560 \uc77c\uc774 \uc788\uc5b4\uc11c, CTAS\ub85c \uc77c\ubd80\ubd84\ub9cc \ubc31\uc5c5\ud574\ub450\uace0 \ud639\uc2dc \ubb38\uc81c\uac00 \uc788\ub294\uc9c0 \ucffc\ub9ac\ub97c \ud574\ubd24\ub294\ub370 \ub2e4\uc74c\uacfc \uac19\uc740 \uc5d0\ub7ec\uac00 \ubc1c\uc0dd\ud588\ub2e4. Error converting column: 126 to DOUBLE Error parsing row: file: hdfs:\/\/cluster01\/user\/hive\/warehouse\/imp.db\/tab_dpcn_info_l\/a7422143a7852306-8b0cbcb600000005-1957752130_data.0, before offset: 629145600 \uc5d0\ub7ec \uba54\uc2dc\uc9c0 \uc790\uccb4\ub294 126\ubc88 \uceec\ub7fc\uc744 DOUBLE \ub85c \ud615\ubcc0\ud658 \ud558\ub294\ub370 \uc2e4\ud328\ud588\uace0 \ud574\ub2f9 \ubb38\uc81c\uac00 \ubc1c\uc0dd\ud55c \ub370\uc774\ud130\uac00 \uc704 hdfs:\/\/.. \uacbd\ub85c\uc5d0, offset \ubc88\uc9f8 \uc904\uc5d0 \uc788\ub2e4\ub294 \ub0b4\uc6a9\uc778\ub370, [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[161],"tags":[],"class_list":["post-1491","post","type-post","status-publish","format-standard","hentry","category-impala"],"_links":{"self":[{"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/posts\/1491","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/comments?post=1491"}],"version-history":[{"count":0,"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/posts\/1491\/revisions"}],"wp:attachment":[{"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/media?parent=1491"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/categories?post=1491"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/oboki.net\/workspace\/wp-json\/wp\/v2\/tags?post=1491"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}