http://www.heavygravity.com/visualodf/refman.pdf (PDF Format) or http://www.heavygravity.com/visualodf/index.html (HTML Format)
This handbook describes the usage of the VisualODF classes and explains how to implement and integrate these items:
The handbook also explains the details of the following items that are related to experimenting with VisualODF:
QT can be installed on either a Windows computer or a Linux distribution for compiling VisualODF. The QT version must be version 4.1 or greater (version 3.x.x will not work without some re-working - please just use the newer versions of QT). Note: It must be noted that QT is slightly easier to install on a Linux computer than a Windows PC with MSVC++. This is only due to the fact that QT can be compiled as-is on a Linux machine, but must be patched to compile properly using the MSVC++ compiler. QT for Windows (the open-source edition) is configured by default to compile with the MingWG [spelling?] compiler. A simple patch allows it to be compiled without problem if you are using the MSVC++ compiler.
Connecting signals and slots between classes is a very simple process, and does not have to be done during initialization. This is useful, for example, if a user wants to switch between several video processing classes that all send signals to display widgets: the signal can be disconnected and re-connected at will depending on which video processing class the user chooses.
All in all, signals and slots allow a very convenient interface for sending and receiving notifications. Signals and slots are not as fast as using other signaling mechanisms (such as mutexes and condition variables) but are perfect for our purposes.
These must all be sucessfully installed before VisualODF can be used. If the example VisualODF program is not going to be compiled and the video playback and capture capability are not needed, then the SDL and ffmpeg libraries do not need to be installed. To install these libraries, please refer to the documentation on the respective websites.
Note: These libraries are cross-platform compatible. Although the sample VisualODF application was only compiled on a Linux platform, it is possible to compile all of the classes on a Windows platform as well.
If you have met the requirements, you must simply navigate to the visualODF directory and type:
qmake
make
and the executable will be built (replace "make" with "nmake" if you are using Windows). To make the plugins, navigate to the src/ directory and then the plugin directory of your choice. Then type:
qmake
make
make install
If you are using Linux, you must have root permissions when you type make install. Again if you are using Windows, use "nmake" instead of "make".
To compile the examples, open a command prompt and navigate to the $QTDIR directory (the directory you installed QT into). There will be an "examples" directory there with a file called 'examples.pro'. In order to create the so-called "Makefile", you must type: qmake in the examples directory. If the executable 'qmake' is not found, then likely you didn't set up the environment variable $QTDIR correctly.
Next, if using Linux, type make to compile all the examples. For windows, type nmake instead. If the examples properly compile (navigate to the various examples and try them out!) then you should have QT properly installed.
It is also essential to understand the purpose of qmake for creating 'Makefiles' to compile your projects, as this should be used when creating QT based projects. The Makefile generated by qmake will ensure that the proper intermediate files are created during compilation of a QT-based class. If the qmake system is not desirable for whatever reason, then please look into the QT MOC (Meta Object Compiler) to determine how to manually compile these intermediate files.
Please read the documentation that comes with QT (by invoking the QT Assistant ("assistant")) for more information on using qmake.
Some knowledge of creating a custom QT widget is needed before creating a widget plugin (refer to the examples in the documentation that comes with QT). Once you have created a custom widget, you must simply create a new class that derives from QDesignerCustomWidgetInterface. The class declaration would look like so:
class piePlugin : public QObject, public QDesignerCustomWidgetInterface
{
Q_OBJECT
Q_INTERFACES(QDesignerCustomWidgetInterface)
public:
piePlugin(QObject *parent = 0);
bool isContainer() const;
bool isInitialized() const;
QIcon icon() const;
QString domXml() const;
QString group() const;
QString includeFile() const;
QString name() const;
QString toolTip() const;
QString whatsThis() const;
QWidget *createWidget(QWidget *parent);
void initialize(QDesignerFormEditorInterface *core);
private:
bool initialized;
};
To see an example implemenation of a Widget Plugin, please look at the class "piePlugin" included with VisualODF.
slots keyword.
public slots: mySlot(int i);
This slot has an integer as a parameter.
A signal is even easier. It is declared like so:
signals:
mySignal(int i);
To connect our signal and slot together, the code looks similar to this:
connect(&sourceObject, SIGNAL(mySignal(int)), &destinationObject, SLOT(mySlot()));
Whenever we use the keyword emit with the signal mySignal, the slot mySlot will be notified. Not only that, but the integer parameter i will be passed between the signal and slot.
This simple but powerful mechanism allows us to pass data between classes easily and easy event notification. In the VisualODF example program, signals and slots are used extensively. The following diagram shows how signals and slots work together in the VisualOSF demo.
If you aren't familiar with the names of the objects in the diagram, that's ok; the class documentation gives plenty of detail on them. Starting from left to right, the seqence of events that occur that of interest in this example are:
frameReady signal: displayImage, which belongs to the Video 1 object, and processFrame which belongs to the Test Algorithm object. They both do different things with this data: the Video 1 object simply displays the data, and the Test Algorithm object will do some type of processing of the image.Test Algorithm in turn has two signals: imageReady(QImage *img) and foundObstacle(...) (where the ... represent several parameters that are omitted for clarity). These are connected to three other kinds of display widgets (Video 2, Pie Widget, and TCAS Widget).foundObstacle(...) signal, and the Video 2 object displays the QImage passed to it (presumably the result of the image processing, in this example).The important points that need to be noted when creating custom processing algorithm:
The image data is stored in a QImage instance. For more information on the QImage class, please open up the QT Assistant and locate the proper reference.
FlightGear has many features, among which are (as taken from the FlightGear website):
FlightGear allows the recording of position and attitude data during flights. This data, combined with a recorded video of the actual flight, could be used as a basis of analysis for obstacle detection. In order to create obstacles in the path of the UAV, one could simply use a second computer and connect to a FlightGear server and position the "obstacle plane" accordingly. Using the autopilot capabilities make scenario creation easier than manual control.
The preceding screen capture shows the view from the cockpit on of the obstacle plane (left windows), and the moving map display showing two red aircraft : one is the obstacle plane, and the second is the UAV (which is flying on a second PC). They are both connected to a FlightGear server.
This is a demonstration of obtaining frames from an existing video source to be displayed and processed with other VisualODF classes included in the demo. The class "TestAlg" (the example image processing algorithm) is able to take these captures frames and invert the RGB values, while the "videoDisplayWidget" simply displays the captured frames.
The class QtFFPlay is responsible for setting up, opening, and acquiring screenshots of a given video stream. It uses the ffmpeg libraries to do this, and should be compatible with any video source that ffmpeg understands. QtFFPlay uses an instance of the class QtFFThread to handle the low-level work related to reading and displaying the video (via the SDL libraries), as well as grabbing video frames.
QtFFPlay is based on the player "ffplay" that is included with the ffmpeg libraries, and shares much code from it. However, ffplay was written in C (not C++), and QtFFPlay can be used within QT (it derives from QWidget). The lack of documentation with the ffmpeg libraries (as well as ffplay) are reflected in the C++ implementations of QtFFPlay and QtFFThread.
Y-Component Layout (30 pixels, or values)
| 1 | 2 | 3 | 4 | 5 | 6 |
| 7 | 8 | 9 | 10 | 11 | 12 |
| 13 | 14 | 15 | 16 | 17 | 18 |
| 19 | 20 | 21 | 22 | 23 | 24 |
| 25 | 26 | 27 | 28 | 29 | 30 |
| 31 | 32 | 33 | 34 | 35 | 36 |
| 1,2,7,8 | 3,4,9,10 | 5,6,11,12 |
| 13,14,19,20 | 9,10,15,16 | 11,12,17,18 |
| 25,26,31,32 | 27,28,33,34 | 29,30,35,36 |
| 1,2,7,8 | 3,4,9,10 | 5,6,11,12 |
| 13,14,19,20 | 9,10,15,16 | 11,12,17,18 |
| 25,26,31,32 | 27,28,33,34 | 29,30,35,36 |
The gray (top) grid represents 30 pixels in a 6x6 image (6 wide, 6 high). Each U and V component corresponds with four Y values. For example, the Y values (or pixels) 1,2,7, and 8 all use the same U value: the first value in the upper left hand corner of the cyan box (with an index of 1). Likewise, the pixels 3,4,9, and 10 belong to the second U value (with an index of 2). The V component uses the same type of layout as well. It is easy to see that this type of image storage uses less space than a 24-bit RGB value : we would use 90 bytes with RGB and only 48 bytes with YUV. This type of layout of YUV information is called YV12, and is a so-called "planar' YUV format.
Other types of YUV formats exist as well, but this seems to be a very commonly used one.
For more information on YUV formats, please see this website: http://www.fourcc.org/fccyuv.htm.
To do the transformation from YUV to RGB, one must apply the following matrix:
For more information on YUV and its relation to RGB, please see the Wikipedia article on YUV [the image above is from that article].
If your image processing algorithm design is sound and works, then you should have a fully functioning application. The example program that comes with VisualODF demonstrates all of these points and should be used as a starting point (please see the main class obstDet).
After a successful application is built, the next step should be to create a custom QT widget and widget plugin for display. Please see the pieDisplay or tcasdisplay for examples of custom QT widgets, and piePlugin, tcasDispPlugin, or videoDisplayPlugin for example widget plugins. All of the code should be documented well enough to allow you to follow along and build your own widgets and plugins.
I did not give detailed information on some of the aspects of using the classes; that is up to the user to read and learn about. Also, some other aspects - such as installing SDL and ffmpeg, and questions like "How do I add a form to my project?" are not covered, as they can be found elsewhere and don't need to be repeated here.
However, if anyone has questions about anything related to this project, please feel free to contact me. My email address is : rob _at_ heavygravity dot com.
Other notes:
1.4.7