The Original ‘Command Line’ Code

We perform a couple of distinct steps in this code. First, we open our directory for iteration with:

And then write the contents of the file into a buffer with:

Not too complicated: We use getline() to only get the raw text of a line, to which we then add a \r\n. We do this because part of our task is to recreate proper tags, and it just so happens that so long as we know where our line breaks are, we can do so reliably. Adding our own line break tags means cross-platform differences will be eliminated, plus our p tag code will run exactly as intended.

The remainder of the code removes extra formatting from headline elements, as well as adds p tags in proper order and sequence. Most of this logic, as it turns out, was the same in Qt, though we did have some significant differences in key locations.

The New Code

The new code segment is the result of a fair bit of experimentation. The command line version was in fact easier to create, though only because of the higher availability of resources for performing file opening, string manipulation, and file writing in plain old C++.

Some key points are as follows:

The method used to iterate through a directory:

…feels a bit strange to me, almost as if I’m missing something. Specifically, the while() statement checks for an empty string if no further files are found. This just seems…wrong to me, but I could be mistaken. For the record, examples given in Qt help documentation had a flaw of never returning the last element of a directory. For example, if we had four items, only three would be iterated through. The do { } while() was the only way I found to hit all files.

As similar as the core logic is, here’s where things take a pretty radical departure from the command line version.

In this code we’re using the handy QTextStream on an address of a QFile object to read each line of text, to which we add the \r\n to, just like the command line version.

We also save a chunk of time here by using REGEX enabled replace method to remove unwanted tags from the headline text:

As you may recall in the old code we used a more complex:

The last part, the section where we add our properly closed p tags, is largely untouched, save for using indexOf() instead of find(), as well as using mid() to build our temporary string for editing.

The last step was to write the fixed file back to the file system. In the first version we used the rather succinct ofstream to do the job, The new version used a more complex series of command as:

Of course we may have a more simplistic way of doing this, but I didn’t find one.

Closing Thoughts

The end result of our project is a slick little Qt desktop app that removed the burden of command line operation for its use. However, as nice as this may be for the end-user, as a developer what I was most curious to see was if Qt would stand up to the challenge of being user friendly, powerful, and at the same time, a joy to work with.

Simple answer: It was. Qt is simply brilliant–I’m sold, hooked, and thoroughly excited to learn and do more with this library. It’s all the power and functionality of C++, but wrapped into a cross platform, easy to use deployment platform with powerful tools and solid documentation.

Though of course this is somewhat relative when it comes to porting command line app to a GUI. Our command line app weighed in at 223 lines, Qt: 368. While it was very easy to find examples of file operation for vanilla C++ on the web, it was markedly more difficult for Qt. In fact, as hinted at in the text I’m still quite unsure of the “properness” of this code. It works well enough, but am I missing something?

Do I really need to used a QTextStream to write data, or should I have used the more low-level write() function inherited from QIODevice? This question reminds me of the difference in PHP between using:

verses using:

In short, one simple step vs. a few more obtuse ones. It’s by no means wrong, but I can’t help but wonder…

Of course it should also be said that this program could use more error and sanity checks if it was to ever be released an a commercial app, but in this case, it’s just for pedagogical purposes.

Complete Application Source Download

Being a Qt app you should be able to run this code (build) on any platform Qt supports.
Download cleaner.zip

The goal during this step is to make exclusive use of the Qt frameworks built-in functions and methods for handling these tasks, as the end result must be a cross-platform compatible app.

We’re already getting a valid directory entry from part 2, so the first step in this part is to figure out how we iterate and query files from our base folder.

As setup for this task, in the first app (the command line one) we defined a few globals:

We’ll keep that basic structure, but of course implement them as member private variables. It should be noted that in the new version the ‘base’ directory value comes from our Worker constructor with:

In the worker constructor we then:

Same thing as the original version really, though one thing I had to check on is if using / will be a problem for Windows. *Turns out the answer is no.*

With the setup tasks done notice that in the GUI we have two buttons: The first allows us to select the base folder, the second kicks off the Process Images task. Notice how we do this: in cleaner.enc__cppqt our clicked() SIGNAL of the processButton button is assigned to the processAction slot, which in turn, after creating our Worker instance object, calls cleanImages(). And so we’re getting an images directory via the GUI–so far so good!

This brings us to the next big question question: how do we open and iterate over directories?

Iterator Over Directories

As you may recall the first task of our program is to remove all files with a _bw_ in the name from the /images sub-folder. It looks like we have: QDirIterator and QDir to help us perform this duty, with the possibly of others as well. The former seems to be for general access, the latter for iterating–we want to iterate.

Basic QDirIterator Implementation

Pretty strait forward really. Iterator over the directory, making sure to skip the path items.

This takes care of the iteration, now we need to find and remove files that have the _bw_.

In the old code we used:

I should step back to mention the first step was to get the file name. In the first attempt we relied on a struct returned by dirent to store file info for checking, now we have a nice simple fileName() method to call. I’m not going to lie, the Qt way feels much cleaner than the command line version. dirent was tricky to work with at first without having a debugger, this is no longer an issue in Qt. With that out of the way, what of finding the text “_bw_”?

I suspected, and was proven right, that QString would have some type of find function. Sure enough:

Beautiful! I can’t help but show my gratitude over such an elegant and natural thing. Without any documentation look up I was able to basically guess and implement my search function in one line. Yes std::string had the same thing with .find(), but this was a nice treat regardless.

Right, so now how do we delete files?

In the first version we had to use a separate variable (location) and a member function .c_str() to pass to the delete(char*) call. This is fine, but a touch verbose. It would be nice to encapsulate this a bit more.

Delete File Implementation

It turns out that we can use the same remove() call, though it wants a char*, we’ve got a QString. As was the goal anyway, we want to use native solutions for cross-platform consistency. I did some digging and found QFile. QFile has a member function for removing files: .remove(). Even better though, QDirIterator has a member function filePath() to return the full path to the file (not just file name). Of course I may have missed similar functionality in the first go (such as using QDir), but now we save ourselves the hassle of rebuilding the file path from scratch. The final code is:

Sure it’s about the same length as the original command line code, if not a bit longer, but it feels more intuitive–almost like OO PHP coding. I do not however, like the idea of creating a new QFile for each item to remove, but I’m learning–I’m sure we’ll find a better way in the next few posts.

Final Method Implementation

For the sake of completeness, here’s the final cleanImages() function:

Up next, rewriting the cleanXMLStoryFiles() function.

At the end of this post we’ll have a shell ready for the actual porting process to being. In other words, a GUI that loads, reacts to button presses, and updates status text.

So here’s the code:

main.enc__cppqt

cleaner.enc__cppqt

cleaner.h

worker.enc__cppqt

worker.h

The biggest thing to note about this code is that we’re not using the Qt Designer program to create the UI. It’s all hand coded, done for the sake of learning the core UI model. If you’re learning Qt I suggest you try this as well. Yes the Qt Designer tool is fantastic and yes I’ll use it for most tasks moving forward, but for this project we’ll keep it low-level.

Core Implementation

On the implementation level it’s important to note that we’re using a totally object oriented approach to the UI and logic. Speaking of UI and Logic, note how we’ve split the app in two logical components: cleaner.enc__cppqt which holds the presentation logic, worker.enc__cppqt for the ‘model’.

On a releated note, as the XML logic will almost certainly get parallelized we’ll take special note of how we add and handle additional classes and data structures. From what I can see Qt may want all parallel code in separate classes, so we’ll see. We may end up editing the core functionality of worker.enc__cppqt, or possibly add a workerXML.ccp to the mix.

Also note how our call to: QFileDialog::getExistingDirectory() via: void Cleaner::loadAction() in cleaner.enc__cppqt is how we get the file path to process. In short, we’ve already replaced the command line apps argument with a GUI based equivalent!

To that end notice that as of right now we still have implementation bugs. So for example we create pointer to an instance of Worker on the heap in processAction() but do not call delete() on it. Of course this is wrong, and we’ll fix that in time–for now though the main idea is this code creates the UI which reacts to member functions of the worker class.

Next Steps

That’s it for the core UI code, in part 2 we’ll start porting our base application.

Go To Part 3 –>
Link

The application in question is a simple tool for cleaning up the export of a Print Publishing CMS system for eventual use in a Web-based CMS. By simple I mean we open a folder and loop through it, diving into sub-folders as needed to:

1. Remove all images tagged with a _bw_
2. Open XML ‘story’ files and remove extra line breaks from headline elements, and clean up content HTML.

The application started as an Apple Xcode project, and has now moved into NetBeans 6.8 on Linux. In the end I hope to learn a good deal about the Qt framework, as well as proper C++ application development.

How and why are we doing this?

Command line apps are only so useful, so the next natural step was to port it to a GUI. I had not done this before, so it would be an education experience. Perhpas you find yourself in a similar situation, stuck with a command line tool when a nice GUI would be preferred. Of course doing so can be intimidating at first, as in most cases the libraries needed to accomplish this task can be quite daunting.

To that end I we’ll use Qt (read: cute) because it’s open source for non-commercial apps, is incredibly powerful, cross-platform, easy to learn, and well documented.

The Code

So what have we got to work with? I did a fair bit of experimentation during the first pass, what we see here is the sanitized version with most comments and junk taken out. That said, it should work on *NIX based distributions, Windows is iffy, and not tested.

The Current Program

The program is broken into several logical sections: The first part of the program takes the user supplied command line argument of a folder and creates a simple variable: basePath. This path is then used to build the other locations of our sub-folder structure. Here we also define a few text constants used in the XML file cleanup step.

With a valid path, we loop though the images folder and delete any files with a _bw_ in the name. The _bw_ is for Black and White, and is a known quantity in this work-flow.

I should add at this point the folder structure we are traversing looks like:

xmlout

–images/

—-/image_files.jog

–null/

–stories/

—-/story.xml

—-/story.xml

The next step is slightly longer, here we need to clean up XML files. The process really isn’t important for the porting process, as we’ll probably end up with very similar code in the end anyway. The long and short of it though is at several places in the XML output are unclosed <p> tags, extra line breaks and formatting, and other general messiness that cause an XML import of these files to fail when going to the ‘target’ system.

The one thing I do want to pay attention to is the file read process. It’s in this area that we’ll probably find the most diversion from the command line app. In particular, I want to see how Qt makes file reading easier, and also, I want to explore parallelization at this step. I can envision reading in a batch of files, say, 50 at a time, and then spawning new threads to handle the clean up tasks.

Of course we then write the cleaned up files back to the system, something that again, could probably be done in concert with a parallel processing model.

More generally though, with a port I want to explore a more object oriented way of performing these steps, as well as adding more error and sanity checks.

Coming In Part 2

Stay tuned for part 2 where we’ll take a look at the ported code I’ve created thus far. We’ll talk about general Qt development, and what goals I have the project in general.

Code Download

A note of caution: This code has not been tested, and may not work on Windows. Of course this is one area a proper Qt port will help with!

main.cpp
Go To Part 2 –>
Link

The following code will spawn 2 threads (for a total of three program threads), and calculate 40 iterations of the fibonacci_number function. You can make it spawn more threads via threads, as well as change the number of iterations via the iterations < 40 condition.

I also have some test code at the top of our threadfunc method for variable passing. Any time you start working with threads you need to be aware of the pitfalls that come with data access, and this is something I very much look forward to learning more about.

Please note the code was run on Linux, specifically Ubuntu 10.04. You’ll need the g++ compiler and other dev libs via individual packages (e.g. libc6-dev) or <sudo aptitude install build-essential>.

Also, as the in-code comment below states, you need to include the Posix Threads library in the Linker  Properties of the project properties area when using NetBeans, I’m not sure about other IDE’s.