For quite some time I’ve been getting duplicate sources.list entry errors when performing software updates/installs on our Ubuntu server. The reason I hadn’t done anything about it was this error is just an annoyance and wasn’t impacting on the servers performance. The problem stems from this server being used for development and testing, so it is always being upgraded to the latest Ubuntu release. This even includes beta releases. The server started out life on Ubuntu 9.04 release and now is sitting on Ubuntu 12.04. 

The error produced “duplicate sources.list entry ” indicates “You may want to run apt-get update to correct these problems”. However running apt-get update just produces the same error over and over again. The problem is fairly obvious form the error, our sources.list has duplicate entries. Just what is the sources.list for? When all else fails read the manual. So the command “man sources.list” tells us this:

“The package resource list is used to locate archives of the package
distribution system in use on the system. At this time, this manual
page documents only the packaging system used by the Debian GNU/Linux
system. This control file is /etc/apt/sources.list.

The source list is designed to support any number of active sources and
a variety of source media. The file lists one source per line, with the
most preferred source listed first. The format of each line is: type
uri args The first item, type determines the format for args. uri is a
Universal Resource Identifier (URI), which is a superset of the more
specific and well-known Universal Resource Locator, or URL. The rest of
the line can be marked as a comment by using a #.”

This basically means the source.list tells the server where to find the software packages which it can install. To fix this problem I thought I was going to have to edit this file manually. The first thing to do is locate it, which was easy enough because the manual tells us where it is located. It can be found at /etc/apt/sources.list. Editing the file I was confronted with a massive conglomerate of commented out entries for various sources intermingled with currently enabled sources. It was a mess all the upgrades and manual edits over the years had turned the sources.list into spaghetti . To manually go through the sources.list and find the duplicates was going to take some time. There must be an easier way and of course there is. This website http://repogen.simplylinux.ch/ presents you with a simple form to select your Ubuntu release and which repositories you wish to include. At the end of the process it produces something like this:

#############################################################
################### OFFICIAL UBUNTU REPOS ###################
#############################################################

###### Ubuntu Main Repos
deb http://au.archive.ubuntu.com/ubuntu/ precise main restricted universe multiverse
deb-src http://au.archive.ubuntu.com/ubuntu/ precise main restricted universe multiverse

###### Ubuntu Update Repos
deb http://au.archive.ubuntu.com/ubuntu/ precise-security main restricted universe multiverse
deb http://au.archive.ubuntu.com/ubuntu/ precise-updates main restricted universe multiverse
deb http://au.archive.ubuntu.com/ubuntu/ precise-proposed main restricted universe multiverse
deb http://au.archive.ubuntu.com/ubuntu/ precise-backports main restricted universe multiverse
deb-src http://au.archive.ubuntu.com/ubuntu/ precise-security main restricted universe multiverse
deb-src http://au.archive.ubuntu.com/ubuntu/ precise-updates main restricted universe multiverse
deb-src http://au.archive.ubuntu.com/ubuntu/ precise-proposed main restricted universe multiverse
deb-src http://au.archive.ubuntu.com/ubuntu/ precise-backports main restricted universe multiverse

###### Ubuntu Partner Repo

deb http://archive.canonical.com/ubuntu precise partner
deb-src http://archive.canonical.com/ubuntu precise partner

###### Ubuntu Extras Repo
deb http://extras.ubuntu.com/ubuntu precise main
deb-src http://extras.ubuntu.com/ubuntu precise main

I can tell you this looks like an object of beauty compared to the original file I started with. All that was left to do was backup our current sources.list (in case this didn’t work) and with a text editor replace the content with the above. To test it works I ran apt-get update again. It worked great and not a single error was seen.