Mirror updates

Bryan Fullerton

10 Nov 2004 10 Nov '04

1:32 p.m.

Howdy, I host the Canadian FTP mirror for OTRS. I've noticed that when I mirror the FTP site each night I get an updated copy of the entire /misc directory. I'm assuming this is because it's being checked out of CVS each day, or some other process is updating the timestamps on all the files. In any case, it's rather irritating, as I get a huge email listing all the updated otrs:/misc files every morning. So, just as a FYI, I'm omitting the /misc directory from my nightly mirror and only updating it once a week on Sunday. Thanks, Bryan

Show replies by date

Eberhard Moenkeberg

10 Nov 10 Nov

1:44 p.m.

Hi, On Wed, 10 Nov 2004, Bryan Fullerton wrote:

...

I host the Canadian FTP mirror for OTRS. I've noticed that when I mirror the FTP site each night I get an updated copy of the entire /misc directory. I'm assuming this is because it's being checked out of CVS each day, or some other process is updating the timestamps on all the files. In any case, it's rather irritating, as I get a huge email listing all the updated otrs:/misc files every morning.

So, just as a FYI, I'm omitting the /misc directory from my nightly mirror and only updating it once a week on Sunday.

It is misc/doc/ only. I do a full mirroring with rsync each night, and so these 25 MB "new files" are matching the "old files" very very good. Number of files: 2034 Number of files transferred: 1592 Total file size: 747797684 bytes Total transferred file size: 25808580 bytes Literal data: 8736 bytes Matched data: 25799844 bytes File list size: 41912 Total bytes written: 216404 Total bytes read: 237288 You see we had to fetch bare 237 kB. And the whole run needs less than one minute. Cheers -e -- Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org)

Bryan Fullerton

1:50 p.m.

On Wed, 10 Nov 2004 14:44:22 +0100 (CET), Eberhard Moenkeberg wrote:

...

I do a full mirroring with rsync each night, and so these 25 MB "new files" are matching the "old files" very very good.

Yes, the FTP mirroring software I'm using is not as smart as rsync, it just goes by timestamp. I should probably be updating using rsync instead, but I haven't set that up before - are you using a specific login on otrs.org, or is there some sort of anon rsync setup there? We can take this offlist if it's not useful for everyone else. :) Thanks, Bryan

Eberhard Moenkeberg

1:56 p.m.

Hi, On Wed, 10 Nov 2004, Bryan Fullerton wrote:

...

On Wed, 10 Nov 2004 14:44:22 +0100 (CET), Eberhard Moenkeberg wrote:

...

...
I do a full mirroring with rsync each night, and so these 25 MB "new files" are matching the "old files" very very good.

Yes, the FTP mirroring software I'm using is not as smart as rsync, it just goes by timestamp. I should probably be updating using rsync instead, but I haven't set that up before - are you using a specific login on otrs.org, or is there some sort of anon rsync setup there? We can take this offlist if it's not useful for everyone else. :)

You can use rsync://ftp.gwdg.de/pub/linux/otrs/ Cheers -e -- Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org)

Eberhard Moenkeberg

2:06 p.m.

Hi once more, On Wed, 10 Nov 2004, Eberhard Moenkeberg wrote:

...

On Wed, 10 Nov 2004, Bryan Fullerton wrote:

...
On Wed, 10 Nov 2004 14:44:22 +0100 (CET), Eberhard Moenkeberg wrote:

...

...
...
I do a full mirroring with rsync each night, and so these 25 MB "new files" are matching the "old files" very very good.

Yes, the FTP mirroring software I'm using is not as smart as rsync, it just goes by timestamp. I should probably be updating using rsync instead, but I haven't set that up before - are you using a specific login on otrs.org, or is there some sort of anon rsync setup there? We can take this offlist if it's not useful for everyone else. :)

You can use rsync://ftp.gwdg.de/pub/linux/otrs/

If you like to migrate your ftp tree to rsync without re-fetching everything, you can do it like the advices in http://ftp.gwdg.de/pub/linux/people/emoenke/rsync.suse_update First step: tell rsync "-vv --dry-run" to let him tell what he would do. Just to follow Lenin - "Vertrauen ist gut, Kontrolle ist besser". Second step: tell rsync "-vv --size-only --existing" to let him verify your contents with help of MD5 sums and correct the naturally crippled ftp time stamps. Third step: do a "regular" rsync. Cheers -e -- Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org)

Bryan Fullerton

2:47 p.m.

On Wed, 10 Nov 2004 15:06:36 +0100 (CET), Eberhard Moenkeberg wrote:

...

If you like to migrate your ftp tree to rsync without re-fetching everything, you can do it like the advices in

http://ftp.gwdg.de/pub/linux/people/emoenke/rsync.suse_update

Ah, too late, I did the whole thing already. :) It's working great, the only thing that's weird is it's printing out errors like: rsync: failed to set permissions on "/home/ftp/pub/otrs/RPMS/suse/9.0": Operation not permitted (1) for each sub-directory (but not the main otrs directory). Not sure why, the entire /home/ftp/pub/otrs directory is owned by the user doing the rsync (user nobody). :/ Thanks! Bryan

Martin Edenhofer

2:02 p.m.

Hi Eberhard, Eberhard Moenkeberg wrote:

...

...
So, just as a FYI, I'm omitting the /misc directory from my nightly mirror and only updating it once a week on Sunday.

It is misc/doc/ only.

I do a full mirroring with rsync each night, and so these 25 MB "new files" are matching the "old files" very very good.

Number of files: 2034 Number of files transferred: 1592 Total file size: 747797684 bytes Total transferred file size: 25808580 bytes Literal data: 8736 bytes Matched data: 25799844 bytes File list size: 41912 Total bytes written: 216404 Total bytes read: 237288

You see we had to fetch bare 237 kB. And the whole run needs less than one minute.

Yes. That's true. rsync is really nice! :) PS: Also thanks for your excellent ftp services Eberhard! :)

...

Cheers -e

Eberhard Moenkeberg

Martin Edenhofer -- ((otrs.de)) :: OTRS GmbH :: Norsk-Data-Str. 1 :: 61352 Bad Homburg http://www.otrs.de/ :: Manage your communication!

Eberhard Moenkeberg

9:08 p.m.

Hi, On Wed, 10 Nov 2004, Martin Edenhofer wrote:

...

PS: Also thanks for your excellent ftp services Eberhard! :)

NO. I do it for me, not for you. Your role is simply to simulate "usage" as a food for my soul. Cheers -e -- Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org)

Martin Edenhofer

1:47 p.m.

Hi Bryan, Bryan Fullerton wrote:

...

I host the Canadian FTP mirror for OTRS. I've noticed that when I mirror the FTP site each night I get an updated copy of the entire /misc directory. I'm assuming this is because it's being checked out of CVS each day, or some other process is updating the timestamps on all the files. In any case, it's rather irritating, as I get a huge email listing all the updated otrs:/misc files every morning.

So, just as a FYI, I'm omitting the /misc directory from my nightly mirror and only updating it once a week on Sunday.

In case use "/misc/doc/". Because just the docs are generated every night. PS: Bryan, thanks for your mirror service! :)

...

Thanks, Bryan

Martin Edenhofer -- ((otrs.de)) :: OTRS GmbH :: Norsk-Data-Str. 1 :: 61352 Bad Homburg http://www.otrs.de/ :: Manage your communication!

7582

Age (days ago)

7582

Last active (days ago)

List overview

Download

8 comments

3 participants

participants (3)

Bryan Fullerton
Eberhard Moenkeberg
Martin Edenhofer

Mirror updates

Bryan Fullerton

Eberhard Moenkeberg

Bryan Fullerton

Eberhard Moenkeberg

Eberhard Moenkeberg

Bryan Fullerton

Martin Edenhofer

Eberhard Moenkeberg

Martin Edenhofer

tags

participants (3)