Ошибка неверный синтаксис для типа integer

Running COPY results in ERROR: invalid input syntax for integer: "" error message for me. What am I missing?

My /tmp/people.csv file:

"age","first_name","last_name"
"23","Ivan","Poupkine"
"","Eugene","Pirogov"

My /tmp/csv_test.sql file:

CREATE TABLE people (
  age        integer,
  first_name varchar(20),
  last_name  varchar(20)
);

COPY people
FROM '/tmp/people.csv'
WITH (
  FORMAT CSV,
  HEADER true,
  NULL ''
);

DROP TABLE people;

Output:

$ psql postgres -f /tmp/sql_test.sql
CREATE TABLE
psql:sql_test.sql:13: ERROR:  invalid input syntax for integer: ""
CONTEXT:  COPY people, line 3, column age: ""
DROP TABLE

Trivia:

  • PostgreSQL 9.2.4

asked Aug 18, 2013 at 10:08

oldhomemovie's user avatar

oldhomemovieoldhomemovie

14.6k13 gold badges64 silver badges98 bronze badges

1

ERROR: invalid input syntax for integer: «»

"" isn’t a valid integer. PostgreSQL accepts unquoted blank fields as null by default in CSV, but "" would be like writing:

SELECT ''::integer;

and fail for the same reason.

If you want to deal with CSV that has things like quoted empty strings for null integers, you’ll need to feed it to PostgreSQL via a pre-processor that can neaten it up a bit. PostgreSQL’s CSV input doesn’t understand all the weird and wonderful possible abuses of CSV.

Options include:

  • Loading it in a spreadsheet and exporting sane CSV;
  • Using the Python csv module, Perl Text::CSV, etc to pre-process it;
  • Using Perl/Python/whatever to load the CSV and insert it directly into the DB
  • Using an ETL tool like CloverETL, Talend Studio, or Pentaho Kettle

answered Aug 18, 2013 at 10:57

Craig Ringer's user avatar

Craig RingerCraig Ringer

304k74 gold badges681 silver badges767 bronze badges

2

I think it’s better to change your csv file like:

"age","first_name","last_name"
23,Ivan,Poupkine
,Eugene,Pirogov

It’s also possible to define your table like

CREATE TABLE people (
  age        varchar(20),
  first_name varchar(20),
  last_name  varchar(20)
);

and after copy, you can convert empty strings:

select nullif(age, '')::int as age, first_name, last_name
from people

answered Aug 18, 2013 at 10:58

Roman Pekar's user avatar

Roman PekarRoman Pekar

106k28 gold badges192 silver badges196 bronze badges

2

Just came across this while looking for a solution and wanted to add I was able to solve the issue by adding the «null» parameter to the copy_from call:

cur.copy_from(f, tablename, sep=',', null='')

answered Sep 19, 2019 at 12:38

helderreis's user avatar

helderreishelderreis

1111 silver badge3 bronze badges

1

I got this error when loading ‘|’ separated CSV file although there were no ‘»‘ characters in my input file. It turned out that I forgot to specify FORMAT:

COPY … FROM … WITH (FORMAT CSV, DELIMITER ‘|’).

answered Mar 11, 2018 at 0:09

Slobodan Savkovic's user avatar

0

Use the below command to copy data from CSV in a single line without casting and changing your datatype.
Please replace «NULL» by your string which creating error in copy data

copy table_name from 'path to csv file' (format csv, null "NULL", DELIMITER ',', HEADER);

answered Dec 31, 2019 at 9:25

Anil's user avatar

AnilAnil

1331 silver badge8 bronze badges

1

I had this same error on a postgres .sql file with a COPY statement, but my file was tab-separated instead of comma-separated and quoted.

My mistake was that I eagerly copy/pasted the file contents from github, but in that process all the tabs were converted to spaces, hence the error. I had to download and save the raw file to get a good copy.

answered Aug 31, 2015 at 15:02

zwippie's user avatar

zwippiezwippie

15k3 gold badges38 silver badges54 bronze badges

CREATE TABLE people (
  first_name varchar(20),
  age        integer,
  last_name  varchar(20)
);

«first_name»,»age»,»last_name»
Ivan,23,Poupkine
Eugene,,Pirogov

copy people from 'file.csv' with (delimiter ‘;’, null »);

select * from people;

Just in first column…..

Feras Al Sous's user avatar

answered Oct 16, 2018 at 12:06

WSchnuble's user avatar

1

Ended up doing this using csvfix:

csvfix map -fv '' -tv '0' /tmp/people.csv > /tmp/people_fixed.csv

In case you know for sure which columns were meant to be integer or float, you can specify just them:

csvfix map -f 1 -fv '' -tv '0' /tmp/people.csv > /tmp/people_fixed.csv

Without specifying the exact columns, one may experience an obvious side-effect, where a blank string will be turned into a string with a 0 character.

answered Aug 18, 2013 at 14:10

oldhomemovie's user avatar

oldhomemovieoldhomemovie

14.6k13 gold badges64 silver badges98 bronze badges

2

this ought to work without you modifying the source csv file:

alter table people alter column age type text;
copy people from '/tmp/people.csv' with csv;

answered Aug 24, 2017 at 18:05

soyayix's user avatar

soyayixsoyayix

1771 silver badge7 bronze badges

0

There is a way to solve «», the quoted null string as null in integer column,
use FORCE_NULL option :

copy table_name FROM 'file.csv' with (FORMAT CSV, FORCE_NULL(column_name));

see postgresql document, https://www.postgresql.org/docs/current/static/sql-copy.html

answered Oct 1, 2018 at 22:14

Charlie 木匠's user avatar

Charlie 木匠Charlie 木匠

2,19419 silver badges19 bronze badges

All in python (using psycopg2), create the empty table first then use copy_expert to load the csv into it. It should handle for empty values.

import psycopg2
conn = psycopg2.connect(host="hosturl", database="db_name", user="username", password="password")
cur = conn.cursor()
cur.execute("CREATE TABLE schema.destination_table ("
            "age integer, "
            "first_name varchar(20), "
            "last_name varchar(20)"
            ");")

with open(r'C:/tmp/people.csv', 'r') as f:
    next(f)  # Skip the header row. Or remove this line if csv has no header.
    conn.cursor.copy_expert("""COPY schema.destination_table FROM STDIN WITH (FORMAT CSV)""", f)

answered Nov 4, 2020 at 16:40

Theo F's user avatar

Theo FTheo F

1,1571 gold badge11 silver badges18 bronze badges

Incredibly, my solution to the same error was to just re-arrange the columns. For anyone else doing the above solutions and still not getting past the error.

I apparently had to arrange the columns in my CSV file to match the same sequence in the table listing in PGADmin.

answered Oct 17, 2020 at 21:13

user3507825's user avatar

user3507825user3507825

4285 silver badges13 bronze badges

I’m trying to import a .csv file to my postgresql DB.
I created a table as follows:

CREATE TABLE accounts
(
    acc_id integer,
    acc_name text,
    website text,
    lat numeric,
    longe numeric,
    primary_poc text,
    sales_rep_id integer
)

Then I used the following command to import the .csv file

COPY accounts(acc_id,acc_name,website,lat,longe,primary_poc,sales_rep_id) 
FROM 'D:accounts.csv' DELIMITER ';' CSV ;

And my .csv file contains the following:

1;Walmart;www.walmart.com;40.23849561;-75.10329704;Tamara Tuma;321500
2;Exxon Mobil;www.exxonmobil.com;41.16915630;-73.84937379;Sung Shields;321510
3;Apple;www.apple.com;42.29049481;-76.08400942;Jodee Lupo;321520

However, this doesn’t work and the following message appear:

ERROR:  invalid input syntax for type integer: "1"
CONTEXT:  COPY accounts, line 1, column acc_id: "1"
SQL state: 22P02

If you have experienced something similar & can create a reproducible example please open a new issue.


When opening an issue, people will be better able to provide help if you provide code that they can easily understand and use to reproduce the problem. This boils down to ensuring your code that reproduces the problem follows the following guidelines:

  • Minimal – Use as little code as possible that still produces the same problem
  • Complete – Provide all parts someone else needs to reproduce your problem in the question itself
  • Reproducible – Test the code you’re about to provide to make sure it reproduces the problem

Minimal

The more code there is to go through, the less likely people can find your problem. Streamline your example in one of two ways:

  1. Restart from scratch. Create a new program, adding in only what is needed to see the problem. Use simple, descriptive names for functions and variables – don’t copy the names you’re using in your existing code.
  2. Divide and conquer. If you’re not sure what the source of the problem is, start removing code a bit at a time until the problem disappears – then add the last part back.

Don’t sacrifice clarity for brevity when creating a minimal example. Use consistent naming and indentation, and include code comments if needed. Use your code editor’s shortcut for formatting code.

Don’t include any passwords or credentials that must be kept secret.

Complete

Make sure all information necessary to reproduce the problem is included in the issue itself.

If the problem requires some code as well as some XML-based configuration, include code for both. The problem might not be in the code that you think it is in.

Use individual code blocks for each file or snippet you include. Provide a description for the purpose of each block.

DO NOT use images of code. Copy the actual text from your code editor, paste it into the issus, then format it as code. This helps others more easily read and test your code.

Reproducible

To help you solve your problem, others will need to verify that it exists.

Describe the problem. «It doesn’t work» isn’t descriptive enough to help people understand your problem. Instead, tell other readers what the expected behavior should be. Tell other readers what the exact wording of the error message is, and which line of code is producing it. Use a brief but descriptive summary of your problem as the title of your question.

Eliminate any issues that aren’t relevant to the problem. If your question isn’t about a compiler error, ensure that there are no compile-time errors.

Double-check that your example reproduces the problem! If you inadvertently fixed the problem while composing the example but didn’t test it again, you’d want to know that before asking someone else to help.

#javascript #node.js #postgresql #express #server

Вопрос:

Надеюсь, что кто-нибудь может помочь со следующей ошибкой экспресс-сервера, поскольку не уверен, что искать и почему этот тип ошибок продолжает появляться. Он использует серверную часть базы данных postgres. Похоже, это также приводит к сбою сервера.

 error: invalid input syntax for type integer: ""
    at Parser.parseErrorMessage (/home/app/server/node_modules/pg-protocol/dist/parser.js:278:15)
    at Parser.handlePacket (/home/app/server/node_modules/pg-protocol/dist/parser.js:126:29)
    at Parser.parse (/home/app/server/node_modules/pg-protocol/dist/parser.js:39:38)
    at Socket.<anonymous> (/home/app/server/node_modules/pg-protocol/dist/index.js:10:42)
    at Socket.emit (events.js:375:28)
    at addChunk (internal/streams/readable.js:290:12)
    at readableAddChunk (internal/streams/readable.js:265:9)
    at Socket.Readable.push (internal/streams/readable.js:204:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23) {
  length: 101,
  severity: 'ERROR',
  code: '22P02',
  detail: undefined,
  hint: undefined,
  position: undefined,
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'numutils.c',
  line: '323',
  routine: 'pg_strtoint32'
}
 

Есть ли что-то конкретное, что мне нужно искать на основе этого error: invalid input syntax for type integer ?

ОБНОВЛЕНО — основываясь на приведенном ниже комментарии, может ли следующий запрос вставки вызвать эту ошибку?

 const insertLogQuery = `
insert into log (log_id, run_id, s_id, q_id, message)
     values ($1, $2, $3, $4, $5)          
`;
const insertLog = await pool.query(insertLogQuery, [ id, null, null, null, reason ]);
 

где столбцы: run_id , s_id и q_id определяются как целые числа.

Если да, может ли кто-нибудь, пожалуйста, сообщить мне, каков наилучший подход для предотвращения этой ошибки на основе этого кода.

Комментарии:

1. ДА. Вы должны искать преобразования из строковых типов данных integer в свои (нераскрытые) запросы. Ваши данные где-то будут содержать пустую строку.

2. @LaurenzAlbe — Я добавил запрос на вставку, который, как я думаю, может вызвать проблему, основываясь на том, что вы упомянули. Не уверен, как это исправить, хотя, если это проблема.

3. Вы должны использовать процесс, известный как «отладка». Выясните, какой столбец и какая инструкция SQL вызывает ошибку, определите фактическую выданную инструкцию SQL и параметры (например, в файле журнала PostgreSQL). И так далее.

The UNION result set data types will be determined by the logic that favours values that are not string literals — it is described in detail in the manual, also note that, without an explicit type specification, string literals are treated as being of type unknown.

Specifically, these rules apply in your case:

5. Choose the first non-unknown input type which is a preferred type in
that category, if there is one.

and

7. Convert all inputs to the selected type. Fail if there is not a conversion from a given input to the selected type.

This basically means that if you have at least one numeric literal value in a particular column (in your case fourth) in your UNIONised query, Postgres will attempt to coerce the values in the same column in other SELECTs into a number, which obviously fails for the character value '2017-01-01'.

Subsequently, if you have at least one character value in that column that cannot be cast into a number, you will have to use explicit character values for the fourth column in all SELECTs:

insert into configurations
(id, key, description, value)
select 1,'key1','D1', '25'
UNION
select 2,'key2','D2', '365'
UNION
select 3,'key3','D3','2017-01-01'

H/T to Andriy M for his comment that prompted me to read the manual more thoroughly.

  • Ошибка невозможно загрузить профиль пользователя windows 10
  • Ошибка неверный синтаксис для типа bigint
  • Ошибка невозможно загрузить av базу
  • Ошибка неверный сигнал апс
  • Ошибка невозможно вызвать invoke или begininvoke