Quantcast

after migration: question about recommended backup procedure

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

after migration: question about recommended backup procedure

Peter Funk
Hello all,

First I would like to say thank you to Anatoly, Bernhard, Ezio,
Ralf and all others who have helped to improve Roundup and to get
1.5.1 released.

With the help of our employees we successfully migrated our roundup
instance from version 0.8.4 (yes, that old!!!) up to version 1.5.1
last week.  During the same step we have migrated our data base
backend from sqlite to mysql.

Now I've a question:

The roundup Administration Guide points out that the
roundup-admin export is **NOT** recommended for backups.
( http://www.roundup-tracker.org/docs/admin_guide.html#tracker-backup )

But in the following paragraph this section gives only clear
advice of what could be done for users of the postgresql
backend.

We had during the past 11 years a cron script running hourly
which simply copied the tracker home directory using the
rsync program.  I'm aware of the fact that this was probably
a little bit risky if a transaction happened to the sqlite db
file during the process of copying of this file.

Now with mysql as our backend I assume we should use mysqldump
to take snapshot backups of the RDBMS.  Am I right?

Best Regards and thanks in advance,
Peter Funk
--
Peter Funk, home: ✉Oldenburger Str.86, D-27777 Ganderkesee
mobile:+49-179-640-8878 phone:+49-421-20419-0 <http://www.artcom-gmbh.de/>
office: ArtCom GmbH, ✉Haferwende 2, D-28357 Bremen, Germany

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Roundup-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/roundup-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: after migration: question about recommended backup procedure

Ralf Schlatterbeck-3
On Tue, Jan 26, 2016 at 04:45:34PM +0100, Peter Funk wrote:
> We had during the past 11 years a cron script running hourly
> which simply copied the tracker home directory using the
> rsync program.  I'm aware of the fact that this was probably
> a little bit risky if a transaction happened to the sqlite db
> file during the process of copying of this file.
>
> Now with mysql as our backend I assume we should use mysqldump
> to take snapshot backups of the RDBMS.  Am I right?

Yes, a combination of rsync (for the tracker code and the messages and
files that roundup puts in the filesystem) and a database dump is the
way to go.

Note also that roundups export format can take *ages* to import for
large databases. I have used roundup export for quite some time as a
backup and did a re-import after every backup. In the end that simply
didn't finish in 24 hours. In addition csv records can get huge and you
need workarounds like "csv_field_size" in the trackers config.ini to
make export/import work for your database...
It's good to convert from one backend to another but too brittle for
regular backup use.

Ralf
--
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   http://www.runtux.com
Reichergasse 131, A-3411 Weidling       email: [hidden email]

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Roundup-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/roundup-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: after migration: question about recommended backup procedure

Tom Ekberg
I have been doing a two-step back, as you suggest, to backup our trackers. Instead of using 'roundup-admin export' I have been using 'tar -pczf' (preserve permissions, create, compress, output file) to capture the files and 'pg_dump --format=tar... && gzip -9...' to capture the PostgreSQL database tables. Each tracker has this pair of files. Adding the output from 'date +%w-%a' (day number of week and 3 character day of week) to the filename pairs gives a 7 day rolling backup. The whole process took 31 minutes to perform this morning on all 33 of our trackers.

Restoring is likewise quick using 'tar -xf' to the appropriate directory and pg_restore to restore the database tables.

Tom Ekberg
Senior Computer Specialist, Lab Medicine
University of Washington Medical Center
1959 NE Pacific St, MS 357110
Seattle WA 98195
work: (206) 598-8544
email: [hidden email]

On Tue, 26 Jan 2016, Ralf Schlatterbeck wrote:

> Date: Tue, 26 Jan 2016 17:41:08 +0100
> From: Ralf Schlatterbeck <[hidden email]>
> To: [hidden email]
> Subject: Re: [Roundup-users] after migration: question about recommended
>     backup procedure
>
> On Tue, Jan 26, 2016 at 04:45:34PM +0100, Peter Funk wrote:
>> We had during the past 11 years a cron script running hourly
>> which simply copied the tracker home directory using the
>> rsync program.  I'm aware of the fact that this was probably
>> a little bit risky if a transaction happened to the sqlite db
>> file during the process of copying of this file.
>>
>> Now with mysql as our backend I assume we should use mysqldump
>> to take snapshot backups of the RDBMS.  Am I right?
>
> Yes, a combination of rsync (for the tracker code and the messages and
> files that roundup puts in the filesystem) and a database dump is the
> way to go.
>
> Note also that roundups export format can take *ages* to import for
> large databases. I have used roundup export for quite some time as a
> backup and did a re-import after every backup. In the end that simply
> didn't finish in 24 hours. In addition csv records can get huge and you
> need workarounds like "csv_field_size" in the trackers config.ini to
> make export/import work for your database...
> It's good to convert from one backend to another but too brittle for
> regular backup use.
>
> Ralf
> --
> Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
> Open Source Consulting                  www:   http://www.runtux.com
> Reichergasse 131, A-3411 Weidling       email: [hidden email]
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
> _______________________________________________
> Roundup-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/roundup-users
>


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Roundup-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/roundup-users
Loading...