pg_dump: [tar archiver] could not write to output file: No space left on device

cPanel, Industry leading hosting platform with world-class support.
Post Reply
raenk
Posts: 5
Joined: February 23rd, 2019, 4:24 am

pg_dump: [tar archiver] could not write to output file: No space left on device

Post by raenk »

Hi,

I'm running backups on a cPanel account that uses PostgreSQL data bases. One of those is 20GB+.

I'm getting "partially completed" backups.

Code: Select all

Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/centos-root   50G   43G  7.7G  85% /
/dev/sda1               1014M  184M  831M  19% /boot
/dev/mapper/centos-home  170G   24G  147G  14% /home
/dev/sdc1                917G  115G  756G  14% /backup
I already change the 'download' and 'workspace' setting of jetbackup to a location on a larger partition (/backup partition). It worked for some days and then happened again. I looked into postgres setting to change the temp path (temp_tablespaces) to the /backup partition as well, but problem persists.

I've heard that the tar format can have size limitations. But not sure if that could be the cause.

This is a log example:

Code: Select all

[22 February 2019 07:09:01 PM -0700] JetBackup Version 4.0.9
[22 February 2019 07:09:01 PM -0700] Creating Incremental Backup for Accounts Configurations, Files, Databases, Emails, Cron Jobs, DNS Zones, SSL Certificates...
[22 February 2019 07:09:01 PM -0700] Preparing accounts list for backup...DONE - 1 Accounts found for backup
[22 February 2019 07:09:01 PM -0700] Preparing progress bar data...DONE
[22 February 2019 07:09:02 PM -0700] 
[22 February 2019 07:09:02 PM -0700] Executing onBackupStart Hooks...
[22 February 2019 07:09:02 PM -0700] Pushing onBackupStart Notification...DONE
[22 February 2019 07:09:02 PM -0700] Pushing onBackupStart to Management Console...DONE
[22 February 2019 07:09:02 PM -0700] onBackupStart Hooks Done
[22 February 2019 07:09:02 PM -0700] Checking connection...DONE
[22 February 2019 07:09:02 PM -0700] 	Checking if destination disk space usage reached to the limit of 95%...DONE - Limit didn't reached - Continuing backup
[22 February 2019 07:09:02 PM -0700] 	Destination Disk Space Usage is 11.03% (106.1 GB out of 961.3 GB)
[22 February 2019 07:09:02 PM -0700] Account "jivabe"
[22 February 2019 07:09:02 PM -0700] 
[22 February 2019 07:09:02 PM -0700] 	Account 1 of 1
[22 February 2019 07:09:02 PM -0700] Executing onBackupAccountStart Hooks...
[22 February 2019 07:09:02 PM -0700] onBackupAccountStart Hooks Done
[22 February 2019 07:09:02 PM -0700] 	Server Load: 0.02
[22 February 2019 07:09:02 PM -0700] 	Checking if destination disk space usage reached to the limit of 95%...DONE - Limit didn't reached - Continuing backup
[22 February 2019 07:09:02 PM -0700] 	Destination Disk Space Usage is 11.03% (106.1 GB out of 961.3 GB)
[22 February 2019 07:09:02 PM -0700] 
[22 February 2019 07:09:02 PM -0700] 	Configuration
[22 February 2019 07:09:02 PM -0700] 		Grabbing...DONE
[22 February 2019 07:09:03 PM -0700] 		Transferring to destination...DONE
[22 February 2019 07:09:03 PM -0700] 
[22 February 2019 07:09:03 PM -0700] 	Files
[22 February 2019 07:09:03 PM -0700] 		Transferring to destination...DONE
[22 February 2019 07:09:10 PM -0700] 
[22 February 2019 07:09:10 PM -0700] 	Databases
[22 February 2019 07:09:10 PM -0700] 		Grabbing...
[22 February 2019 07:09:10 PM -0700] 			Dumping Database 'jivabe_evaluaciones' (Engine: PostgreSQL)...DONE
[22 February 2019 07:09:23 PM -0700] 			Dumping Database 'jivabe_pruebas_walmart' (Engine: PostgreSQL)...DONE
[22 February 2019 07:09:39 PM -0700] 			Dumping Database 'jivabe_demo' (Engine: PostgreSQL)...DONE
[22 February 2019 07:09:39 PM -0700] 			Dumping Database 'jivabe_promomecanicos' (Engine: PostgreSQL)...FAILED
[22 February 2019 07:12:04 PM -0700] [ERROR] pg_dump: [tar archiver] could not write to output file: No space left on device

[22 February 2019 07:12:04 PM -0700] 			Dumping Database 'jivabe_inmuebles_walmart' (Engine: PostgreSQL)...DONE
[22 February 2019 07:12:11 PM -0700] 			Dumping Database 'jivabe_demo_inmuebles' (Engine: PostgreSQL)...DONE
[22 February 2019 07:12:25 PM -0700] 			Dumping Database 'jivabe_flex' (Engine: PostgreSQL)...DONE
[22 February 2019 07:12:26 PM -0700] 			Dumping Database 'jivabe_test' (Engine: PostgreSQL)...DONE
[22 February 2019 07:15:18 PM -0700] 		Transferring to destination...DONE
[22 February 2019 07:19:29 PM -0700] 
[22 February 2019 07:19:29 PM -0700] 	Emails
[22 February 2019 07:19:29 PM -0700] 		Fetching email accounts list...DONE - 7 Accounts found
[22 February 2019 07:19:29 PM -0700] 		Transferring to destination...DONE - Already handled by the Files section
[22 February 2019 07:19:29 PM -0700] 
[22 February 2019 07:19:29 PM -0700] 	Cron Jobs
[22 February 2019 07:19:29 PM -0700] 		Transferring to destination...DONE - Already handled by the Configuration section
[22 February 2019 07:19:29 PM -0700] 
[22 February 2019 07:19:29 PM -0700] 	DNS Zones
[22 February 2019 07:19:29 PM -0700] 		Preparing...DONE - 1 DNS Zones found
[22 February 2019 07:19:29 PM -0700] 		Transferring to destination...DONE - Already handled by the Configuration section
[22 February 2019 07:19:29 PM -0700] 
[22 February 2019 07:19:29 PM -0700] 	SSL Certificates
[22 February 2019 07:19:29 PM -0700] 		Transferring to destination...DONE - Already handled by the Configuration section
[22 February 2019 07:19:30 PM -0700] 
[22 February 2019 07:19:30 PM -0700] 	Clearing old snapshots...DONE - 0 snapshots deleted
[22 February 2019 07:19:30 PM -0700] Executing onBackupAccountEnd Hooks...
[22 February 2019 07:19:30 PM -0700] onBackupAccountEnd Hooks Done
[22 February 2019 07:19:30 PM -0700] 	Time spent on this account: 10 Minutes and 28 Seconds
[22 February 2019 07:19:30 PM -0700] 	0% is done from the total progress
[22 February 2019 07:19:30 PM -0700] 
[22 February 2019 07:19:30 PM -0700] 	100% is done from the total progress
[22 February 2019 07:19:30 PM -0700] 
[22 February 2019 07:19:30 PM -0700] Executing onBackupEnd Hooks...
[22 February 2019 07:19:30 PM -0700] Pushing onBackupEnd Notification...DONE
[22 February 2019 07:19:30 PM -0700] Pushing onBackupEnd to Management Console...DONE
[22 February 2019 07:19:30 PM -0700] onBackupEnd Hooks Done
[22 February 2019 07:19:30 PM -0700] Job Partially completed
What else should I look into?

Thanks
ofir
Staff Member
Staff Member
Posts: 40
Joined: June 14th, 2018, 10:03 am

Re: pg_dump: [tar archiver] could not write to output file: No space left on device

Post by ofir »

Hi Raenk,

Thank you for sharing this with us, I will try to assist you as much as possible.
I will need some screenshots from your JetBackup settings please -

Settings -> General tab
Settings ->Resource tab

Thank you,
Ofir, JetApps Team.
raenk
Posts: 5
Joined: February 23rd, 2019, 4:24 am

Re: pg_dump: [tar archiver] could not write to output file: No space left on device

Post by raenk »

Sure, here they are:
Image Image Image Image
Thanks!
ofir
Staff Member
Staff Member
Posts: 40
Joined: June 14th, 2018, 10:03 am

Re: pg_dump: [tar archiver] could not write to output file: No space left on device

Post by ofir »

Hi,

Thank you for the information.

It seems like that the PGSQL engine is still keeping temp files on your "/" partition, during the dump process.

Please execute manual 'pg_dump' into the backup partition and in parallel see if any temp files are saved in the "/" partition.

The command should be something like: ```pg_dump -U postgres DATABASE -F t -f /backup/```

Thanks
raenk
Posts: 5
Joined: February 23rd, 2019, 4:24 am

Re: pg_dump: [tar archiver] could not write to output file: No space left on device

Post by raenk »

Hi,

Yes, I still get "no space left on device". I found this information:
https://dba.stackexchange.com/questions ... or-pg-dump

Which basically says that pg_dump tar format option does that (creates temporary tables). I tried the "plain" format and worked.

Is there a way to change the file format use for pg_dump to plain instead of tar?

Thanks
ofir
Staff Member
Staff Member
Posts: 40
Joined: June 14th, 2018, 10:03 am

Re: pg_dump: [tar archiver] could not write to output file: No space left on device

Post by ofir »

Hi,

We were able to confirm that the `pg_dump` tar flag uses the tar temp path.
I have opened an internal case for changing the PostgreSQL dump format.
The fix will probably release in JetBackup 4.0.13

Again, thank you for the information.

Regards.
raenk
Posts: 5
Joined: February 23rd, 2019, 4:24 am

Re: pg_dump: [tar archiver] could not write to output file: No space left on device

Post by raenk »

Hi Ofir,

Do you have an ETA for the release?

Thanks
ofir
Staff Member
Staff Member
Posts: 40
Joined: June 14th, 2018, 10:03 am

Re: pg_dump: [tar archiver] could not write to output file: No space left on device

Post by ofir »

Hi,

The release can take about few weeks (JetBackup 4.0.12 was just released yesterday to EDGE tier).

If you need a faster fix for that, please open a support ticket and provide your server login credentials, and we do our best to manually implement the fix on your server once we solve it.

Regards,
Ofir, JetApps team.
raenk
Posts: 5
Joined: February 23rd, 2019, 4:24 am

Re: pg_dump: [tar archiver] could not write to output file: No space left on device

Post by raenk »

Hi,

Thanks for your help and offer. I'll move the postgresql data directory to a different partition.
Post Reply