Delete Duplicate Rows/Records in MySQL Table

Most articles on removing duplicate rows from a MySQL table involve 3 steps, but the following query is what I use for purging dupe records in one simple query.

DELETE FROM `myTable` WHERE id NOT IN (SELECT t1.id FROM (SELECT id, groupByColumn FROM `myTable` ORDER BY id DESC) as t1 GROUP BY t1.groupByColumn)

– “myTable” is the name of the table with duplicate rows
– “id” is the name of the primary key identifier in “myTable”
– “groupByColumn” is the name of the column used to differentiate records as duplicates

Example: Table of Videos with the duplicate match being made on the “title” field.

DELETE FROM `videos` WHERE id NOT IN (SELECT t1.id FROM (SELECT id, title FROM `videos` ORDER BY id DESC) as t1 GROUP BY t1.title)

It’s a good SQL query to save or bookmark for those times when you need do some maintenance or cleanup during development.

How to Copy or Move a Joomla Website

Intro

If you manage one or more Joomla websites, eventually you’ll have to move them elsewhere. It’s pretty much fact. Performance requirements will change, you’ll find better pricing elsewhere, your dedicated server died, etc.

There are a few options most people have to choose from:

  • FTP client and phpMyAdmin method (aka the long, boring method)
  • SSH/Shell method (aka the cool, quick method)
  • PHP system() method (aka middle of the road and kind of fun method)
  • Joomla “clone” component (your mom could do it)

Move Joomla with an FTP Client and phpMyAdmin

  1. Download the entire Joomla website via FTP client (you’re using S-FTP to connect, right?)
  2. Use phpMyAdmin to export a SQL dump of your database
  3. Upload the entire Joomla website via FTP client
  4. Use phpMyAdmin on the new server to import the SQL dump from the old website
  5. Update configuration.php:
    1. Update the MySQL database credentials
    2. Update the tmp/logs path
    3. If you use FTP Layer, update the credentials
  6. Update .htaccess to match any changed server requirements

Easy and straightforward. Long, slow process, but any Jr. Network Admin  could handle this for you if you don’t want to get your hands dirty.

Clone Joomla with SSH (shell) Access

  1. Login to your server via SSH
  2. Browse to your Joomla website root
  3. Run these commands:
    [code lang=”bash”]tar -czf ../backup-example-com-20090619.tar.gz .

    mv ../backup-example-com-20090619.tar.gz ./

    mysqldump -u yourUsername -p -h yourMySQLHostname yourDatabaseName > backup-example-com-20090619.sql[/code]

  4. Do you need to move this to a remote server or another location on the same server?
    1. Local Path
      1. Copy both backup files to the new website root
      2. Browse to the new Joomla website root
    2. Remote Path
      1. Login to remote server via SSH
      2. Browse to the new Joomla website root
      3. Use wget to download the archive and SQL dump to this server:
        [code lang=”bash”]wget http://www.example.com/backup-example-com-20090619.tar.gz
        wget http://www.example.com/backup-example-com-20090619.sql[/code]
  5. Run this command:
    [code lang=”bash”]tar -xzf backup-example-com-20090619.tar.gz[/code]
  6. Run this command (assuming you have made a new, blank database)
    [code lang=”bash”]mysql -u yourNewUsername -p -h yourNewMySQLHost yourNewDatabase  < backup-example-com-20090619.sql[/code]
  7. Update configuration.php & .htaccess as shown in the first example

More complicated (obviously), but if you like doing things the hard fun way, then it’s a great way to go.

Using PHP’s system() or back-tick Commands to Copy Joomla Website

I wasn’t made aware of this method until after I started managing a whole slew of websites in a cloud hosting platform (Scale My Site). Cloud hosting (and many shared hosting platforms) do not provide access to SSH because it’s simply not feasible. Cloud hosting in particular due to your website running across hundreds of different server nodes. You can perform the same functions as the SSH procedure above using system execution commands in PHP.

  1. Create a new file called copy-me.php
  2. Write the following code into this file:
    [code lang=”php”]echo `tar -czf ../backup-example-com-20090619.tar.gz . && mv ../backup-example-com-20090619.tar.gz ./`;
    echo `mysqldump -u yourUsername -p -h yourMySQLHostname yourDatabaseName > backup-example-com-20090619.sql`;[/code]
  3. Execute the PHP file by accessing it from a browser:
    Browser> http://www.example.com/copy-me.php
  4. Create a new file on the destination website called update-me.php
  5. Write the following code into this file:
    [code lang=”php”]echo `wget http://www.example.com/backup-example-com-20090619.tar.gz`;
    echo `wget http://www.example.com/backup-example-com-20090619.sql`;
    echo `tar -xzf backup-example-com-20090619.tar.gz`;
    echo `mysql -u yourNewUsername -p -h yourNewMySQLHost yourNewDatabase  < backup-example-com-20090619.sql`;[/code]
  6. Execute the PHP file by accessing it from your browser:
    Browser> http://www.exampleDestination.com/update-me.php
  7. Update the configuration.php and .htaccess files as needed

Cool, huh?

Use a Backup or Clone Component from Joomla Extension Directory

If you prefer not to do anything yourself, and want to keep it as simple as possible, then a backup component from the JED is the way to go:

http://extensions.joomla.org/extensions/access-&-security/backup

I have only used one of those components before, and I found that there were a few bugs needing to be worked out, and it ended up taking more time to do the backup, move, and clone that I needed to do than when I did so manually.

Foreward

There are shortcuts you can take here depending on your environment. For instance, you never need to create archives at all, as you can pipe the mysqldump output directly to another mysql command (with the new database’s credentials). However, I prefer to use archives and solid files especially when using PHP-based method, because you could end up accidentaly accessing the cloner file and wiping an existing MySQL database (if you aren’t careful). So, on top of all this, I’d recommend removing the update-me and copy-me files after using them.

Tips on Load Balancing a Joomla Cluster with HAProxy

For the past several weeks, I have been working with Joomla in a clustered environment. We have a single load-balancer running HAProxy that sends requests to two web servers synchronized with unison. One server is a hybrid and includes both the MySQL database as well as Apache2/PHP5. The other web server is strictly Apache2/PHP5. We have been renting two super fast dedicated servers temporarily until we acquire some new hardware, so I had to make do with what few servers I had.

Update: Having written this blog post almost a full year ago, I have since then completely switched all of my Joomla websites to the automatically scaling website cloud host: Scale My Site. Since doing so, we haven’t had to deal with HAProxy, load balancing, or anything with regard to scaling due to the hosting cloud’s seamlessly clustered environment. I highly recommend anyone reading this article right now to check out cloud hosting to get load balancing/scaling for your Joomla website without breaking a sweat.

The load balancer is located at our own colo. I followed the tutorial on Setting Up A High-Availability Load Balancer (With Failover and Session Support) With HAProxy/Heartbeat On Debian Etch to set up two servers at our colo in an ActivePassive fashion using Heartbeat for redundancy.

Weighted Load Balancing

Since I’m using only two web servers and one needs to serve database requests, I decided to set weights in HAProxy so that the hybrid server receives half as many requests as the dedicated web server. Here is an example of what my haproxy.cfg file contains:

/etc/haproxy.cfg

global
    log 127.0.0.1 local0
    log 127.0.0.1 local1 notice
    maxconn 4096
    user haproxy
    group haproxy

defaults
    log global
    mode http
    option httplog
    option dontlognull
    retries 3
    redispatch
    maxconn 2000
    contimeout 5000
    clitimeout 50000
    srvtimeout 50000

listen webfarm 63.123.123.100:80
    mode http
    balance roundrobin
    cookie SERVERID insert nocache indirect
    option forwardfor
    option httpchk HEAD /check.txt HTTP/1.0

    # Stats
    stats enable
    stats auth admin:password

    # Web Node
    server SBNode1 63.123.123.101:80 cookie Server1 weight 20 check
    # Web + MySQL Node
    server SBNode2 63.123.123.102:80 cookie Server2 weight 10 check

How to Use the Administrator Control Panel in a Joomla Cluster

Many people understand that it’s a super big pain to work with the administrator control panel in a Joomla clustered environment. First of all, you’ll keep getting kicked out every few page requests, even while using sticky/persistent load balancing. Second, working with backend WYSIWYG rich-text editors is nearly impossible. I figured out how to do it, and here’s what I did.

  1. Decide upon the management node
  2. Give the management node a public host entry in DNS (e.g. node1.yourdomain.com)
  3. Open configuration.php for editing
  4. Locate the “live site” variable ($mosConfig_live_site)
  5. Replace with “http://” . $_SERVER[“HTTP_HOST”];
  6. Save

Using the current host as the live site allows you to use node1.yourdomain.com as an access point for the control panel. You can work in the control panel without doing this, but you will run into tons of problems with rich-text editors and custom components that request the live site URL in their underlying code.

Update: Recently, I implemented a load balancing solution using HAProxy that used the ACL system to send all traffic with /administrator/ in the URL to one “master” node, and it provided a way around the Joomla configuration change mentioned above. Check out this blog post for more info.