pages tagged sysadmin

Warning about low disck space

Sun, 25 Oct 2020 00:50:16 +0000

For whatever reason, even if my Macbook Pro computer booted with more than 10 GB disk space available, after working on it and having it powered on constantly for a day or two, its disk space very often will fall dramatically and sometimes even almost fill up with nominally something like 200 MB left. And it often causes issues:

Having to "manually" kill applications that are no longer responsive using Activity Monitor.
Some applications not properly exiting and saving status and metadata. For example, Eclipse workspace data, backup files in Emacs, and currently opened tabs in Google Chrome.

I made a Bash script to check and warn about low disk space and have it run periodically as a cron job.

中国古代书籍分类

Fri, 08 May 2020 20:51:23 +0000

中国古代书籍分类

中国古代书籍可以定义为1912年之前出版印行及1912年之后影印等形式的同类电子图书。
经部
- 易、书、诗、三礼礼制、乐、春秋、四书、孝经、小学、群经总义、谶纬、丛编、石经等
史部
- 正史別史、编年、纪事本末、杂史、载记地理、史表、史评史抄、传记、政书职官、诏令奏议、金石、目录、丛编等
子部
- 儒家、道家、兵家、法家、农家、医家、杂家、小说家、天文历算、术数、艺术工艺、丛编等
集部
- 楚辞、别集、总集、诗文评、词、曲、小说、丛编等
丛部
- 类书通类、类书专类、杂纂丛书、辑佚丛书、郡邑丛书、氏族丛书、独撰丛书、丛书补缺等

TODO

补充各类书籍实例。

Backup MediaWiki

Tue, 16 May 2017 23:59:39 +0000

Here is a memo on backing up MediaWiki instances, say deployed as a part of a Web site mywebsite.com.

Here is a listing of concrete steps:

Get inside the backup root directory on local file system:

cd /Volumes/BACKUP/mywebsite.com

Backup using the `backup_mediawiki.sh` backup script

Login web server
Update VCS repository https://github.com/lumeng/MediaWiki_Backup

Back up using MediaWiki_Backup/backup_mediawiki.sh

 # assuming web directory is ~/mywebsite.com/wiki
 WIKI_PATH="mywebsite.com/wiki"
 # assuming the path to save a subdirectory backup_YYYYMMDD created by backup is path/to/backup/mywebsite.com/wiki
 WIKI_BACKUP_PATH="path/to/backup/mywebsite.com/wiki"
 # get to the home path before start
 cd
 # Start backup. This will create backup path/to/backup/mywebsite.com/wiki/backup_YYYYMMDD.
 path/to/backup_mediawiki.sh -d $WIKI_BACKUP_PATH -w $WIKI_PATH

Rsync the backup to a local hard drive:

cd /Volumes/BACKUP/mywebsite.com

Backup the whole web site user's home directory that includes the backup files created above, using rsync



rsync --exclude-from rsync_backup_exclusion.txt -thrivpbl user@webhost.com:/home/websiteuser rsync_backup/

Ideally, upload the backup to cloud storage such as Drobpox.

HTML backup using `wget` for immediate read

Optionally, one can also keep a crawled version of a MediaWiki instances. Sometimes, it can be useful to have a copy of HTML files for immediate read offline.

cd /Volumes/BACKUP/mywebsite.com/wget_backup
mkdir mywebsite.com-wiki__wget_backup_YYYYMMDD
cd mywebsite.com-wiki__wget_backup_YYYYMMDD
# crawl the whole Web site
# wget -k -p -r -E http://www.mywebsite.com/
# crawl the pages of the MediaWiki instance excluding the Help and Special pages
wget -k -p -r --user-agent='Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2049.0 Safari/537.36' -R '*Special*' -R '*Help*' -E http://www.mywebsite.com/wiki/
cd ..
7z -a -mx=9 mywebsite.com-wiki__wget_backup_YYYYMMDD.7z wget_backup_YYYYMMDD

Remarks:

-k: convert links to suit local viewing
-p: download page requisites/dependencies
-r: download recursively
--user-agent: set "fake" user agent for the purpose of emulating regular browsing as sometimes site checks user agent. Check user agent string at useragentstring.com.

As for time cost to create the wget-crawled backup, for reference, it took about 30 min to download a small MediaWiki installation with hundreds of user-created pages in an experiment I did.

If there is a small set of pages that you need to backup, curl may be alternatively used, for example,

# download multiple pages
curl -O http://mywebsite.com/wiki/Foo_Bar[01-10]

References

https://www.mediawiki.org/wiki/Manual:Backing_up_a_wiki
https://www.mediawiki.org/wiki/Fullsitebackup
https://www.mediawiki.org/wiki/Manual:DumpBackup.php
https://wikitech.wikimedia.org/wiki/Category:Dumps

Back up MediaWiki

Tue, 16 May 2017 23:59:39 +0000

Wrote a note about backing up MediaWiki including its database (MySQL or SQLite), content pages exported as an XML file, all images, and the entire directory where MediaWiki installed, which includes LocalSettings.php and extensions that usually contain customization.

pages tagged sysadmin

Warning about low disck space

中国古代书籍分类

中国古代书籍分类

TODO

Backup MediaWiki

Backup using the backup_mediawiki.sh backup script

Backup the whole web site user's home directory that includes the backup files created above, using rsync

HTML backup using wget for immediate read

References

Back up MediaWiki

Backup using the `backup_mediawiki.sh` backup script

HTML backup using `wget` for immediate read