Delete Duplicate Rows/Records in MySQL Table

Most articles on removing duplicate rows from a MySQL table involve 3 steps, but the following query is what I use for purging dupe records in one simple query.

DELETE FROM `myTable` WHERE id NOT IN (SELECT t1.id FROM (SELECT id, groupByColumn FROM `myTable` ORDER BY id DESC) as t1 GROUP BY t1.groupByColumn)

– “myTable” is the name of the table with duplicate rows
– “id” is the name of the primary key identifier in “myTable”
– “groupByColumn” is the name of the column used to differentiate records as duplicates

Example: Table of Videos with the duplicate match being made on the “title” field.

DELETE FROM `videos` WHERE id NOT IN (SELECT t1.id FROM (SELECT id, title FROM `videos` ORDER BY id DESC) as t1 GROUP BY t1.title)

It’s a good SQL query to save or bookmark for those times when you need do some maintenance or cleanup during development.

How to Copy or Move a Joomla Website

Intro

If you manage one or more Joomla websites, eventually you’ll have to move them elsewhere. It’s pretty much fact. Performance requirements will change, you’ll find better pricing elsewhere, your dedicated server died, etc.

There are a few options most people have to choose from:

  • FTP client and phpMyAdmin method (aka the long, boring method)
  • SSH/Shell method (aka the cool, quick method)
  • PHP system() method (aka middle of the road and kind of fun method)
  • Joomla “clone” component (your mom could do it)

Move Joomla with an FTP Client and phpMyAdmin

  1. Download the entire Joomla website via FTP client (you’re using S-FTP to connect, right?)
  2. Use phpMyAdmin to export a SQL dump of your database
  3. Upload the entire Joomla website via FTP client
  4. Use phpMyAdmin on the new server to import the SQL dump from the old website
  5. Update configuration.php:
    1. Update the MySQL database credentials
    2. Update the tmp/logs path
    3. If you use FTP Layer, update the credentials
  6. Update .htaccess to match any changed server requirements

Easy and straightforward. Long, slow process, but any Jr. Network Admin  could handle this for you if you don’t want to get your hands dirty.

Clone Joomla with SSH (shell) Access

  1. Login to your server via SSH
  2. Browse to your Joomla website root
  3. Run these commands:
    [code lang=”bash”]tar -czf ../backup-example-com-20090619.tar.gz .

    mv ../backup-example-com-20090619.tar.gz ./

    mysqldump -u yourUsername -p -h yourMySQLHostname yourDatabaseName > backup-example-com-20090619.sql[/code]

  4. Do you need to move this to a remote server or another location on the same server?
    1. Local Path
      1. Copy both backup files to the new website root
      2. Browse to the new Joomla website root
    2. Remote Path
      1. Login to remote server via SSH
      2. Browse to the new Joomla website root
      3. Use wget to download the archive and SQL dump to this server:
        [code lang=”bash”]wget http://www.example.com/backup-example-com-20090619.tar.gz
        wget http://www.example.com/backup-example-com-20090619.sql[/code]
  5. Run this command:
    [code lang=”bash”]tar -xzf backup-example-com-20090619.tar.gz[/code]
  6. Run this command (assuming you have made a new, blank database)
    [code lang=”bash”]mysql -u yourNewUsername -p -h yourNewMySQLHost yourNewDatabase  < backup-example-com-20090619.sql[/code]
  7. Update configuration.php & .htaccess as shown in the first example

More complicated (obviously), but if you like doing things the hard fun way, then it’s a great way to go.

Using PHP’s system() or back-tick Commands to Copy Joomla Website

I wasn’t made aware of this method until after I started managing a whole slew of websites in a cloud hosting platform (Scale My Site). Cloud hosting (and many shared hosting platforms) do not provide access to SSH because it’s simply not feasible. Cloud hosting in particular due to your website running across hundreds of different server nodes. You can perform the same functions as the SSH procedure above using system execution commands in PHP.

  1. Create a new file called copy-me.php
  2. Write the following code into this file:
    [code lang=”php”]echo `tar -czf ../backup-example-com-20090619.tar.gz . && mv ../backup-example-com-20090619.tar.gz ./`;
    echo `mysqldump -u yourUsername -p -h yourMySQLHostname yourDatabaseName > backup-example-com-20090619.sql`;[/code]
  3. Execute the PHP file by accessing it from a browser:
    Browser> http://www.example.com/copy-me.php
  4. Create a new file on the destination website called update-me.php
  5. Write the following code into this file:
    [code lang=”php”]echo `wget http://www.example.com/backup-example-com-20090619.tar.gz`;
    echo `wget http://www.example.com/backup-example-com-20090619.sql`;
    echo `tar -xzf backup-example-com-20090619.tar.gz`;
    echo `mysql -u yourNewUsername -p -h yourNewMySQLHost yourNewDatabase  < backup-example-com-20090619.sql`;[/code]
  6. Execute the PHP file by accessing it from your browser:
    Browser> http://www.exampleDestination.com/update-me.php
  7. Update the configuration.php and .htaccess files as needed

Cool, huh?

Use a Backup or Clone Component from Joomla Extension Directory

If you prefer not to do anything yourself, and want to keep it as simple as possible, then a backup component from the JED is the way to go:

http://extensions.joomla.org/extensions/access-&-security/backup

I have only used one of those components before, and I found that there were a few bugs needing to be worked out, and it ended up taking more time to do the backup, move, and clone that I needed to do than when I did so manually.

Foreward

There are shortcuts you can take here depending on your environment. For instance, you never need to create archives at all, as you can pipe the mysqldump output directly to another mysql command (with the new database’s credentials). However, I prefer to use archives and solid files especially when using PHP-based method, because you could end up accidentaly accessing the cloner file and wiping an existing MySQL database (if you aren’t careful). So, on top of all this, I’d recommend removing the update-me and copy-me files after using them.

How-to Backup Joomla! 1.5 to Amazon S3 with Jets3t

Introduction to backing up a Joomla website to Amazon S3 storage using Jets3t.

We all know backups are important. I’ve found what I consider a pretty good backup solution using Amazon S3. It’s super cheap, your backups are in a secure location, and you can get to them from anywhere. For my backup solution, I’m using Debian Linux (Etch), but this whole setup is not dependent on your current favorite flavor of Linux because it uses Java.

  1. Signup for Amazon S3: http://aws.amazon.com/s3/
  2. Install the latest Java Runtime Environment: http://java.sun.com/javase/downloads/index.jsp
  3. Download Jets3t: http://jets3t.s3.amazonaws.com/downloads.html
  4. Extract Jets3t installation to a location on your server.Example: /usr/local/jets3t/
  5. Add your AWS account key and private key to the “synchronize” tool configuration file:Example: /usr/local/jets3t/configs/synchronize.properties
  6. Use an S3 browser tool like Firefox S3 Organizer to add two buckets: one for file backups and one for MySQL backups.
  7. Add a MySQL user whose primary function is dumping data. Let’s call it ‘dump’ with the password ‘dump’:
    [code lang=”bash”]mysql>GRANT SELECT, LOCK TABLES ON exampleDB.* to ‘dump’ identified by ‘dump’;[/code]
  8. Build your backup script (replace paths with your own) called s3backup.sh:
    [code lang=”bash”]JAVA_HOME=/usr/local/j2re1.4.2_17
    export JAVA_HOME
    JETS3T_HOME=/usr/local/j3ts3t
    export JETS3T_HOME
    SYNC=/usr/local/jets3t/bin/synchronize.sh
    WWWROOT=/var/www/fakeuser/
    MYSQLBUCKET=example-bucket-mysql
    WWWBUCKET=example-bucket-www
    MYSQLDUMPDIR=/usr/local/mysql-dumps
    WWWDUMPDIR=/usr/local/www-dumps
    # Perform backup logic
    dayOfWeek = `date +%a`
    dumpSQL=”backup-www-example-com-${dayOfWeek}.sql.gz”
    dumpWWW=”backup-www-example-com-${dayOfWeek}.tar.gz”
    mysqldump -u dump -pdump exampleDB | gzip > “${MYSQLDUMPDIR}/${dumpSQL}”
    # Compress the website into an archive
    cd ${WWWROOT}
    tar -czf “${WWWDUMPDIR}/${dumpWWW}” .
    # Perform Jets3t synchronize with Amazon S3
    $SYNC –quiet –nodelete UP “${WWWBUCKET}” “${WWWDUMPDIR}/${dumpWWW}”
    rm -f “${WWWDUMPDIR}/${dumpWWW}”
    $SYNC –quiet –nodelete UP “${MYSQLBUCKET}” “${MYSQLDUMPDIR}/${dumpSQL}”
    rm -f “${MYSQLDUMPDIR}/${dumpSQL}”[/code]
  9. Make sure your script has execute permission
  10. Add a cron job to perform daily backups:
    [code lang=”bash”]$>crontab -e
    0 0 * * * /root/s3backup.sh[/code]

That’s it. Good luck!

Tips on Load Balancing a Joomla Cluster with HAProxy

For the past several weeks, I have been working with Joomla in a clustered environment. We have a single load-balancer running HAProxy that sends requests to two web servers synchronized with unison. One server is a hybrid and includes both the MySQL database as well as Apache2/PHP5. The other web server is strictly Apache2/PHP5. We have been renting two super fast dedicated servers temporarily until we acquire some new hardware, so I had to make do with what few servers I had.

Update: Having written this blog post almost a full year ago, I have since then completely switched all of my Joomla websites to the automatically scaling website cloud host: Scale My Site. Since doing so, we haven’t had to deal with HAProxy, load balancing, or anything with regard to scaling due to the hosting cloud’s seamlessly clustered environment. I highly recommend anyone reading this article right now to check out cloud hosting to get load balancing/scaling for your Joomla website without breaking a sweat.

The load balancer is located at our own colo. I followed the tutorial on Setting Up A High-Availability Load Balancer (With Failover and Session Support) With HAProxy/Heartbeat On Debian Etch to set up two servers at our colo in an ActivePassive fashion using Heartbeat for redundancy.

Weighted Load Balancing

Since I’m using only two web servers and one needs to serve database requests, I decided to set weights in HAProxy so that the hybrid server receives half as many requests as the dedicated web server. Here is an example of what my haproxy.cfg file contains:

/etc/haproxy.cfg

global
    log 127.0.0.1 local0
    log 127.0.0.1 local1 notice
    maxconn 4096
    user haproxy
    group haproxy

defaults
    log global
    mode http
    option httplog
    option dontlognull
    retries 3
    redispatch
    maxconn 2000
    contimeout 5000
    clitimeout 50000
    srvtimeout 50000

listen webfarm 63.123.123.100:80
    mode http
    balance roundrobin
    cookie SERVERID insert nocache indirect
    option forwardfor
    option httpchk HEAD /check.txt HTTP/1.0

    # Stats
    stats enable
    stats auth admin:password

    # Web Node
    server SBNode1 63.123.123.101:80 cookie Server1 weight 20 check
    # Web + MySQL Node
    server SBNode2 63.123.123.102:80 cookie Server2 weight 10 check

How to Use the Administrator Control Panel in a Joomla Cluster

Many people understand that it’s a super big pain to work with the administrator control panel in a Joomla clustered environment. First of all, you’ll keep getting kicked out every few page requests, even while using sticky/persistent load balancing. Second, working with backend WYSIWYG rich-text editors is nearly impossible. I figured out how to do it, and here’s what I did.

  1. Decide upon the management node
  2. Give the management node a public host entry in DNS (e.g. node1.yourdomain.com)
  3. Open configuration.php for editing
  4. Locate the “live site” variable ($mosConfig_live_site)
  5. Replace with “http://” . $_SERVER[“HTTP_HOST”];
  6. Save

Using the current host as the live site allows you to use node1.yourdomain.com as an access point for the control panel. You can work in the control panel without doing this, but you will run into tons of problems with rich-text editors and custom components that request the live site URL in their underlying code.

Update: Recently, I implemented a load balancing solution using HAProxy that used the ACL system to send all traffic with /administrator/ in the URL to one “master” node, and it provided a way around the Joomla configuration change mentioned above. Check out this blog post for more info.

Dreamhost Hosting Review

Soon after I opened my own Dreamhost account, my company needed an external hosting environment for Joomla instances, and I recommended Dreamhost. I mentioned it supported unlimited domains, MySQL databases, etc., and had a large amount of space/bandwidth for any of our sub-projects within the company. They even have a One-click install for Joomla! It sounded like a low-cost solution, and seemed to be perfect for what we needed.

…or so we thought.

Continue reading Dreamhost Hosting Review