USB Auto Update Guide

Guest_Jim_* - 2013-04-20 08:51:23 in Gadgets
Category: Gadgets
Reviewed by: Guest_Jim_*   
Reviewed on: May 2, 2013

Introduction:

I, like many others, am somewhat lazy. If I can find a way to simplify my life by automating some process, I will do it even if it takes an amount of effort to begin. Well, I guess one could say that because I have put in the effort you now do not have to, or at least not as much effort.

Before I joined the staff here at OCC, I asked a question on the forums about how to use the Windows Task Scheduler to trigger a batch file upon connecting a USB drive. That batch file would contain instructions on how to update files on the USB drive, so I would not need to manually copy files myself. Well, since I developed that first system, the scripts have evolved into one of the more convoluted things I have ever created... but it works! Also, every one of those convolutions is actually needed and add to the system's overall strength, but they do get complicated. Do not do anything in this guide unless you are an advanced computer user. Honestly, I cannot imagine how anything I explicitly tell you to do could mess up your computer, but you should still be careful.

This guide is going to have you work with batch files, possibly the Event Viewer, definitely the Task Scheduler, and maybe Environment Variables. The latter depends on your computer situation. Also, all of this is being done on a Windows 7 machine, so if you are running a different version of Windows, the location of things may be different.

Batch Files:

Batch files are scripts that can contain a series (or batch) of instructions which can be executed within the Command Prompt as well as some only useful from within a script, such as GOTO and labels, both of which we will be using. They are actually very useful files and are worth understanding.

Event Viewer

The Event Viewer is exactly what its name says. When just about anything happens on your computer, the event is logged by the system and this is where you can view the logs. While viewing these logs it is possible to assign a task to them. The task begins whenever the selected event occurs.

Task Scheduler:

Again, the name gives this one away. This is the system you can use to schedule a task to occur, such as running a program, sending an email, and displaying a message. Without a doubt, we will be visiting this later.

Environment Variables:

Computer variables, like algebraic variables, stand in for other values. When used within a batch file, variables usually stay there and are not recorded by the system. Environment Variables however are accessible to everything on the computer, and many programs use and create them. You may not need them to get your USB Update system to work, but they are required for me.

My Situation:

Since I've mentioned my situation and setup a few times, and will continue to mention it later on, I think I should explain what makes my setup special, and why it requires the USB Update system to be so convoluted. I have two computers, a laptop and a desktop, with user folders on different drives. The laptop keeps my user folder on the C drive while my desktop keeps it on the E drive. The different drive letters means that the paths to any file are also different. Further, I have and use multiple USB drives of different sizes and purposes. This is why I must use Environment Variables and some other tricks, which will be described later; so no matter the computer or the drive, the files can be appropriately updated.

With that explained, let's get to it!

Creating the Update Instructions:

The first step we are going to do can be done at almost any point, so why not do it now? Also, it happens to be the simplest.

We are going to create the batch files with the update instructions. Batch files can be written by any text editor, such as Notepad, and have the '.bat' extension. The first thing to do is identify the files and folders you wish to update. For this guide I am going to use a folder of utility programs I keep on my USB drives and the folder in which I store all of my news items.

There are a few different copy commands that a batch file can use, and for this I recommend using the very powerful, ROBOCOPY, or Robust File Copy. Importantly it has the ability to copy just newer files. Other commands, such as XCopy also have this ability, but ROBOCOPY is better designed to work with directories than XCopy.

The flags we will be using with ROBOCOPY are /mir, /xo, /xf, /xd, and possibly /e. Before I get to those, the format of ROBOCOPY is:

robocopy "Source" "Destination"

That means that to copy over my utilities and news item folders the commands will look like this:

robocopy "%DataDrive%\USB Items\Utilities" "U:\Utilities"

robocopy "%JimContent%\Documents\OCC\News Articles" "U:\OCC\News Articles"

Remember how I mentioned my need for Environment Variables? This is their first appearance as both 'DataDrive' and 'JimContent' are Environment Variables. The '%' symbols tells the system to call the variable. I need the variables here because the location of the folders are on different drives between my laptop and desktop. As Environment Variables are local to the specific computer, the same variable name can have different values.

Truthfully, there is another variable I have to use in those commands, but I have not shown it here because we have not gotten to where I am comfortable explaining its use.

As the commands sit right now, they will instruct every file within the named folders to be copied to the USB drive, instead of just those that need to be updated, and no subfolder will be copied. To copy over subfolders we need to use the /e flag. To only copy over newer, updated files, instead of everything, we use the /xo command, which stands for 'exclude old.'

For my purposes, and likely yours, we are not going to use the /e flag, in this case. Instead we are going to use the /mir flag, which has the same effect, and more. This flag stands for 'mirror' and will make the destination folder mirror the source, by copying over subfolders and deleting anything at the destination but not at the source. As I want the USB drive to represent what is on the computer and not be an ever growing backup, I want the /mir flag to remove what I no longer need.

 

robocopy /mir /xo "%DataDrive%\USB Items\Utilities" "U:\Utilities"

robocopy /mir /xo "%JimContent%\Documents\OCC\News Articles" "U:\OCC\News Articles"

As I said earlier, I have multiple USB drives of different sizes. Let us say this is one of the smaller drives, so I want less to be copied onto it. In that case we will use the /xd and /xf flags which stand for 'exclude directory' and 'exclude file' respectively. These commands do recognize wild cards.

robocopy /mir /xo "%DataDrive%\USB Items\Utilities" "U:\Utilities" /xd "Defrag" "Uninstall"

robocopy /mir /xo "%JimContent%\Documents\OCC\News Articles" "U:\OCC\News Articles" /xf "*.wlx" /xd "*No USB"

I do not actually have directories labeled 'Defrag' and 'Uninstall' but I did not want to name the programs I am actually excluding on my smaller drive. As you can see though, we are able to just list what we want excluded one after another. The 'wlx' files were something you are unlikely to see again as the program that used them is no longer supported. Because there are no spaces in the names we do not need the quotation marks, but I use them out of habit.

At this point, you should be able to go through and generate the batch file(s) you will want to use. To test the batch file(s) I suggest inserting the USB drive you want to update, making sure the destination uses its drive letter, and run the batch file with the command prompt. This way you will be able to watch as the instructions are carried out, and see if any error occurs. Also, be sure you are using 'dummy' files and folders; if you accidentally setup the instructions to delete files, you will not lose anything important.

Creating the Task:

There are two ways to create the USB Update task. One is to find the correct event in the Event Viewer and use the 'Attach Task To This Event' command. The other way, which is easier, is to use the XML file given later in this guide. As you will not learn as much from just importing the file, I am also going to go through the long way, because I care.

First step is to open the Event Viewer. It can be found within the Control Panel under 'System and Security,' 'Administrative Tools,' 'View event logs.' Alternatively, you can just search for it.

Once you open it, you want to follow the tree down 'Applications and Services Logs,' 'Microsoft,' 'Windows,' and then find the 'DriverFrameworks-UserMode,' folder. In there, you want to find the event with Event ID 2101, which I believe is the event associated with Windows giving the USB drive power.

Now you can either Right-Click the event or click the button on the right side of the viewer labeled 'Attach Task To This Event.' Now a window should pop up to direct you through the process of creating a new task. The first step is to name and describe it:

The second step is already filled out because we are attaching the task to the event. The third step is to select what the task is going to do. We want 'Start a Program.'

Now we need to give it a program to run, so I have given it "E:\USB Update.bat." The quotes are important here as there is a space in the name. If you are only using a single USB drive, you should be able to have the task just run the file with all of the update instructions. If you have multiple USB drives or if Windows likes to change the drive letter of the USB drive, the task will have to run a different batch file, which I will explain later in this guide.

The final step just shows you the options so you can fix any mistake.

Once you finish creating the task through the Event Viewer, you can leave and close it. To make any changes to the task you will need to open the Task Scheduler and find it under 'Event Viewer Tasks.'

If you want to import the XML file instead of manually creating the task from an event, open the Task Scheduler and use either the button on the right labeled 'Import Task...' or select it under the Action drop down menu. Just navigate to the XML file and open it. I have removed the information from it that would connect it to my computers. It does however use an Environment Variable: DataDrive. This is because I keep the USB Update.bat file on different drives between my two computers. You can just replace the variable with the correct drive letter or use your own variable. Don't know how to create an Environment Variable? Don't worry, that's the next step and the start of our trip down the rabbit hole.

USB Updater.zip (Contains the XML file to be imported.)

Creating Environment Variables

To create an Environment Variable, you have to get to your system properties, which is really easy. Just open My Computer and click the 'System properties' button on the top. From there click 'Advanced system settings' on the left, which will open a window with an 'Environment Variables...' button on the bottom. Clicking that will open another window that shows you all of the User and System variables. I put my variables under the User variables, but I do not believe it matters which you use; just click the 'New?' button. Now you will get another window asking for the variable name and value. I just use two: DataDrive which is 'E:' and 'JimContent' which is 'E:\Users\Jim,' the location of my user folder on my desktop. On my laptop the values reference the C drive instead of the E drive.

That's all there is to creating an Environment Variable! Just remember that to use the variable you must surround it with '%' signs. Now for the confusing fun part.

Enabling Multiple Computers and Multiple USB Drives (Part 1):

Typically a batch file is a relatively simple set of instructions that are executed from beginning to end. This is not a typical situation though, so we are going to be using a batch file that jumps around itself, and there are reasons for using such a file.

The primary reason I found myself having to do this is that Windows will forget what drive letter it gave a USB drive, so what was J may become K, and that will mess up the updating script. To fix that, we have to use something that will check multiple drive letters. There may be a more elegant way of doing it than I am using, but this works:


@echo off

set var=0

:plugcheck

@echo off

set /a var=%var%+1

if %var% EQU 1 goto plug1

if %var% EQU 2 goto plug2

if %var% EQU 3 goto plug3

if %var% EQU 4 goto plug4

goto:EOF

:plug1

set Drive=I:

goto deviceexist

:plug2

set Drive=J:

goto deviceexist

:plug3

set Drive=K:

goto deviceexist

:plug4

set Drive=L:

goto deviceexist

Yes, that is a lot of syntax to throw at you, but here is what it means. The '@echo off' command suppresses the batch file from throwing up information I don't want to see. You do not have to use it if you do not want to.

'set var=0' creates the variable 'var' and gives it the initial value of 0. The ':plugcheck' and anything else with a single colon in front of it is a label, which a 'goto' command can send the system to. The 'goto:EOF' command sends the system to the end of the file, effectively exiting the script. The 'if' commands operate as you would expect; if the first thing is true, do this. In this case it is if the variable 'var' equals 1, 2, 3, or 4, goto labels 'plug1,' 'plug2,' 'plug3,' or 'plug4.' At those labels, another variable is created and defined; the 'Drive' variable is set to a drive letter, and we are sent to another label, 'driveexist,' which I will explain next.

Enabling Multiple Computers and Multiple USB Drives (Part 2):

:deviceexist

if exist "%Drive%\USB Update.bat" goto existYes

if exist "%Drive%\NO USB Update.bat" goto existNo

goto plugcheck

:existYes

set filename=USB Update

goto devicename

:existNo

set filename=NO USB Update

goto devicename

There is actually quite a lot going on in this segment. First the script checks if there is a 'USB Update.bat' on the USB drive. This is something I had not explained before, because it did not need to be explained. My USB drives contain the file 'USB Update.bat,' which contains the instructions of what files and folders to update. They have different names on my computers, which I will get to in a moment. This is actually a remnant of how I first created this updating system, where the file on the USB drive was simply run and done, so having them all named the same simplified things.

Now, however, the system is different as I keep the update instructions on my computers as well. This is actually useful for redundancy (if your drive malfunctions you still have the instruction set to rebuild it) and for editing, as you can then edit the instructions without the drive attached. Of course, if you edit the instructions, you will need to be able to update the file on the drive, which I will explain soon.

The different filename variable values, 'USB Update' and 'NO USB Update' are to allow me to still automatically update the instruction file on a USB drive without having to run the instruction set. Updating the instruction set on the USB device is what the next bit of code does, and it is both important and complicated, so I am going to break it up into three parts within this guide.

First, the segment that gets the USB device's name, which is stored as a comment within its update-instruction set.

:devicename

for /F "usebackq delims=::" %%B in ("%Drive%\%filename%.bat") do (

@echo on

    echo %%B

@echo off

    ::exit /b

    set device=%%B

    goto batupdate

)

That 'for' command is currently configured to read the 'USB Update.bat' or 'NO USB Update.bat' on the USB drive, depending on which file is indeed on the drive, which was determined earlier. The name of the device, such as 'My 4GB,' is commented at the beginning of the instruction by having two colons in front of it, like this '::My 4GB.' (Of course you do not want to include the period in the comment.) Because the delimiter is set to be two colons, when the command reads that comment, it does not include the colons when it sets variable 'B' to the device name it just read.

The 'echo %%B' command causes the batch file to display the name of the device it is reading. This command is not needed, but can be useful so I have it there. As I do not want the next commands shown, I turn echo off again.

The 'device' variable is able to take its value outside of the 'for' command, unlike the 'B' variable, which is why it is used.

Next, the 'batupdate' label, which updates the instruction set on the USB device:

 

:batupdate

@echo on

xcopy /y /d "%Drive%\%filename%.bat" "%DataDrive%\USB Items\%device% USB Update.bat"

xcopy /y /d "%DataDrive%\USB Items\%device% USB Update.bat" "%Drive%\%filename%.bat"

@echo off

goto update

 

Here we use the XCopy command instead of ROBOCOPY just because. We could use ROBOCOPY if we wanted, but XCopy works fine here. The '/y' flag answers yes to the question of if we want to replace a file when we copy. The '/d' flag is like the '/xo' flag for ROBOCOPY and makes sure the newer version of the file is copied over the older, and not the other way around. As this is a process I may want to monitor, I turned 'echo' on to see it, and then off again when it has completed.

Now we get to actually running the update instructions:

:update

echo Updating %device%

if exist "%Drive%\USB Update.bat" (

call "%DataDrive%\USB Items\%Device% USB Update.bat" Drive

)

goto plugcheck

First we see that it will tell us what device is being updated. Next, if the drive contains the file 'USB Update.bat' it will call the same file located on my computer. (It could just as easily call the file from the USB drive, but I'm funny and wanted it to call the one from the computer.) The 'call' command allows one batch file to execute another and will wait for it to complete before continuing itself. It is extremely important to include the 'Drive' at the end of the call command because that passes the 'Drive' variable to the called batch file. This way the update instruction knows the correct destination of the files to be copied to the USB drive.

After updating the USB drive, it returns to the 'plugcheck' label explained earlier. This way if multiple USB drives are connected, it will update each of them instead of just the first one it finds.

That's it! That is how you automatically update files on one USB drive of many you own, from any computer you own, even if files are in different places on the different computers. Now, do me a favor and never ask me how long it took to get all of this to work right.

To illustrate what the program goes through, I have embedded an annotated graphic on the next page. It does not follow all of the intricacies of the script, but should help make clear what the general process is through the file.

Conclusion:

Here is a graphic of what this USB Auto Update system does, to help you understand its complexities. I have also created an interactive version which provides the code and a brief explanation, but there is no easy way to embed the file within this article. Click the link under the graphic to download the original SVG file, which can be viewed in at least Firefox and Chrome.

 

 

 

 

 



 

Graphic for Web.zip (Contains the interactive SVG.)

 

It is amazing how much work some of us will go through to produce something that enables us to be lazy. I probably spent more time learning how to do all of this and troubleshooting it than I have ever spent manually updating files on a USB drive, but then that is the price we have to pay to be lazy sometimes.

Anyway, this setup is quite possibly more complicated than most of you will need because many of you may not use with multiple computers, like I do, or use multiple USB drives, like I do. As complicated as it may be though, it does work and works well. Just be careful when adapting it for yourself so you do not accidentally delete important files.

Good luck, and cheers to the power of programming (even relatively simple programming)!