If you want to mirror a phpbb or other CMS that ask for an authentication, you can get the coockis from the firefox dir.
In my case is:
cp /home/gnuton/.mozilla/firefox/xxxxxx.default/cookies.sqlite .
Then convert the coockies file from sqlite to plain paper with this script
gnuton@iron:/tmp/SS$ cat SS
#!/bin/bash
#
sqlite3 $1 <<EOF
.mode tabs
.header off
select host as domain,
case substr(host,1,1)=’.’ when 0 then ‘FALSE’ else ‘TRUE’ end as flag,
path,
case isSecure when 0 then ‘FALSE’ else ‘TRUE’ end as secure,
expiry as expiration, name, value from moz_cookies;
EOF
The you have your plaintext cookies file and you can run now wget to do the job.
$ wget –load-cookies cookies.txt -m -np http://mywebsite.com
Well, in linux the best tool to take a copy of a entire website is WGet.
Scenario: To get all the pages, images and the other contents that are in a hypothetic book/ directory of a hypothetic website (eg: example.com), we can run:
wget -m -np -p http://www.example.com/book/
where “m” stands for “mirror”, “np” is “no parent” (we want only the pages in the book/ directory) and the “p” flags tells to wget taking all the resources (like images) needed to display correctly the web pages mirrored in offline mode.
Most people run a home server behind a router. If you have a dynamic IP, this could be very annoying. The server don’t know about the public IP (of the router), the router sometime don’t update the dinamic DNS. The OpenDNS settings are not valid anymore if your IP chages.
To avoid all these issues, this night I wrote a very cool script that updates DynDNS and OpenDNS at the same time.
DynDNS allows me to have a domain like my-server.dyndns.org with a dinamic IP. OpenDNS instead after a free registration and after that I added its DNS IP to my DHCP router configuration, it offers a safe browsing to my network users protecting them from several internet threats like phishing, adult sites ecc..
My server runs this script every hours via cron.
#! /bin/bash
user=MYUSER
password=MYPASSWORD
hostname=MYDNS.MYDNS.ORG
#wildcard allowed values are NOCHG, Y, N
wildcard=NOCHG
mx=NOCHG
# Y=Yes, use it as my primary mail relay. N=No, use it as backup MX record.
backmx=NOCHG
#current IP
currentIP=$( curl -s http://checkip.dyndns.org/ | grep -o “[[:digit:].]\+” )
#check
if [ "$currentIP" != "$(cat lastIP)" ]; then
echo IP changed to $currentIP
echo “$currentIP” > lastIP
#update OpenDNS
/usr/bin/curl -m 60 -u $user:$password ‘https://updates.opendns.com/account/ddns.php?’
#update DynDNS
curl -v -k -u $user:$password “https://members.dyndns.org/nic/update?hostname=$hostname&myip=$currentIP&wildcard=$wildcard&mx=$mx&backmx=$backmx”
else
echo IP not changed. It is still $currentIP
fi
