Roadmap & Status

The public API is simply the architecture, without providing any specific actual codecs, formats, or players. This is mostly done. We also implement some internal com.sun and com.ibm classes as needed to support existing projects which use JMF.

Progress.csv (available in HTML and for download as an CSV file from here) shows the progress of individual (public) classes. Mostly complete apart from GUI and a few RTP classes.

FMJ can be swapped out for JMF when running several Essential JMF samples (their sample multi-image and text players)

Implement javax.media.bean.playerbean.*

Replacements for more internal com.sun and com.ibm classes

Flesh out few JMF classes that are still stubbed out.

This is FMJ's answer to JMStudio, a sample application which demonstrates many of the features of FMJ/JMF.

Currently supports playback (to the extent that the formats/codecs are supported)

Volume control

Playback of RTP audio and video

Transmission of RTP audio using RTP/ULAW

Registry editor

Transmission of RTP audio using other formats

Transmission of RTP video, and audio/video simultaneously

Transcode wizard

FMJ Studio menu options: maintain aspect ration, full screen, snapshot, plugin viewer, capture control, transmission stats

Better and more graphical controls for playback/rate/volume, and graphical controls for things like packet size, etc. There are some controls in EJMF that we may be able to use as a reference.

This is the simplest way to playback video. Native players do not process the media in Java, but simply wrap a native player, and use that player's GUI for display. Internal codecs and such are not exposed, and no JMF-style filter graph is built.

We have Java-based player/handlers using Quicktime, DirectShow, and GStreamer (via GStreamer-Java) wrappers. Each of these is a class named Handler which implements Player.

Some features are missing for some players, including the ability to get the correct media time, seek, fast forward, rewind, etc.

The next step with these wrappers needs to be done, which is to get the bits of the decoded audio/video, to allow them to be pumped through a filter graph, rather than rendered natively. (We would still want to keep the ability to render natively of course)

JMF defines an architecture with for playback/processing with DataSources, Demultiplexers, Codecs, and Renderers. For media playback, a filter graph is built using these elements.

FMJ has implemented basic filter graph construction and execution.

Many audio Demultiplexers / Cocecs are implemented using JavaSound.

A number of RTP Packetizers / Depacketizers / Codecs are implemented (JPEG, ULAW, ALAW, Speex, iLBC)

Sound and video renderers are implemented, with volume control

Playback of streaming audio, such as from internet radio

Play most uncompressed audio files using JavaSound.

Play MP3/OGG files in a GUI using javazoom / jlayer SPIs for JavaSound (FMJ has a JavaSound parser, codec, and renderer). WAV, AU, and AIFF files play with javasound using no SPIs

Alternatively, play OGG files using JFFMPEG

Support FOBS4JMF if in classpath

Support ffmpeg demux/decode via ffmpeg-java (see below)

Filter graph processing needs to improve synchronization of audio and video.

Filter graph processing needs to use multiple threads and double-buffering to improve performance.

More pure-Java Demultiplexers and Codecs need to be implemented

More wrappers around native Demultiplexers and Codecs need to be implemented

Controls on various plugins, such as packet size, etc.

Support native/NIO buffers for improved performance

Video capture device support provided via LTI-CIVIL (DirectShow, Quicktime, V4L2).

Audio capture using a javasound datasource.

Support querying and changing of resolution/format

Support faster throughput (double-buffering)

Support format details in video capture URLs

We have a contributed RTP library, and supporting Codec / DataSource for JPEG/RTP reception and rtp:// URL handling, but integration and testing is still in progress. SIP-Communicator has some packetizers/depacketizers we have borrowed/integrated.

A number of RTP Packetizers / Depacketizers / Codecs are implemented (JPEG, ULAW, ALAW, Speex, iLBC)

Display RTP/JPEG streams, using the rtp:// URL syntax (e.g. rtp://192.168.1.1:8000)

Transmit and playback a number of different RTP audio streams: RTP/ULAW, RTP/ALAW, RTP/Speex, RTP/iLBC

RTP/JPEG transmission (packetizer)

Implementation of other RTP Packetizers / Depacketizers / Codecs, either in pure Java or by wrapping native code.

Synchronization of audio and video

Basic construction and execution of Processors is working. This always means the construction of a filter graph, and often involves Multiplexers at the output end of the graph.

A JavaSound Multiplexer has been written which can write files such as .wav and .au

Advanced features of processing such as inserting codecs into the codec chain, and adding renderers

Multiplexers which wrap native libraries (DirectShow, Quicktime, GStreamer, FFMPEG)

The existing FFMPEG wrappers have shortcomings: (FOBS4JMF: an unnecessary C++ wrapper is in between Java and ffmpeg, hiding functionality) (JFFMPEG: attempt to port FFMPEG to Java, lags behind FFMPEG). So a new FFMPEG wrapper, FFMPEG-Java is being developed using JNA. It is in the FMJ CVS repository, under a different root folder, FFMPEG-Java. It also has its own file releases on the downloads page.

Demultiplexer wraps FFMPEG if available as dynamic libraries.

Supports arbitrary JMF/FMJ DataSources, not limited to file://, http:// etc, supported by FFMPEG

Break out codecs from demultiplexer to be implemented by JMF-style Codecs

Break out software scaling (including YUV/RGB conversion) from demultiplexer to be implemented by JMF-style Codecs

Test/integrate Other ffmpeg protocols: pipe, tcp, rtp

Support encoding

Theora occupies a unique place in the world of media: it is the only modern, performant codec that is unencumbered by patent and royalty issues. It then makes sense that an open-source media project should have first-class support for this. In particular, it would be good have a pure java version of this, since it would be perfect for applets. Cortado could serve this purpose, but it is GPL, not LGPL.

The actual ogg container should already be supported by FMJ's support of Ogg/Vorbis audio.

The first task would be to wrap the native theora library using JNA, has started, see below. The next effort would be to actually port the C to Java.

A theora/ogg/vorbis wrapper, theora-java is being developed using JNA. It is in the FMJ CVS repository, under a different root folder, Theora-Java. It also has its own file releases on the downloads page.

Create a demultiplexer/codecs based on Theora-Java

Support encoding

Miscellaneous tasks

Sample media. It would be great to find public domain media files that we can use and redistribute for testing. We have a few in our samplemedia directory.

Help get JFFMPEG to work with FMJ. Some of the parsers and codecs are now close to working. Much of this work is on the JFFMPEG end.

JMF-based open-source projects: Searching on Sourceforge for JMF turns up a number of open-source projects based on JMF. We should work to make FMJ be able to be used instead of JMF for these. The first step is usually just trying to build the other projects against FMJ instead of JMF. This will turn up various dependencies on Sun and IBM internal libraries, for example. See here for FMJ's list of JMF-related projects.

JMF sample code: We should work to make FMJ be able to be used instead of JMF for various sample code files/projects. The first step is usually just trying to build them against FMJ instead of JMF. This will turn up various dependencies on Sun and IBM internal libraries, for example. See here for FMJ's list of JMF sample code.

• Support capture devices on Mac OSX, Linux 64-bit (JMF only supports Win32, Linux x86, and Solaris)
• Support hotplugging of capture devices
• Support a standard way of adding MIME types. This is supported in JMF using an internal Sun class.
• Native codecs on various platforms, with dynamic determination of supported formats, rather than having to use registry.
• Integration with codec downloading where supported (DirectShow, WMP, Quicktime)
• An API to get tags from various files like MP3/Ogg
• m3u/playlists
• ip cameras, and server end of an ip camera
• CD audio
• data sink for internet radio streaming

I believe the JMF APIs should be updated as well. Although FMJ should comply with official APIs, it may be able to influence the direction of API changes.

It should function 100% Java with no native libraries required, but if native libraries exist, certain features use them.

Here are some of many suggestions (although some of them may be better placed in jffmpeg or FOBS):

• Use of nio buffers instead of JMF's native buffer hack. Sun's current JMF code checks if a buffer is a special native implementation. This is specific to Sun's JMF implementation so the native abilities may not be re-used by other plugins. Ultimately there should be an API change to javax.media.Buffer to use nio.Buffers
• Add a few good renderers. I have written a JOGL renderer that does pretty well, especially when scaling the video image. I have also written an Xv video renderer for rendering YUV directly on Linux systems.
• Being able to choose a particular audio Mixer for audio input or output, and control of line level. Use current sample audio APIs.
• Better video capture DataSources: DirectShow on Win, V4L2 on Linux, IEEE1394 capture on Linux, Quicktime on MacOS (fobs may have this already).
• A well managed registry - possibly using XML or some other editable text format instead of a binary format. API that allows registering mime-types and protocol packages.
• An online registry of JMF plugins where the FMJ may download and install required/missing plugins.
• Good automatic detection of media devices available to FMJ.
• Updated security/permissions handling, using up to date JAAS. Currently if the libraries for JMF and Java3D are in different locations, security exceptions occur.
• Accelerated player/filter implementaions: e.g. GStreamer (using GST-Java), DirectShow.
• Efficient RTP transmitting and receiving: optional native library acceleration.
• Perhaps a MIDI data source. Use current Java MIDI APIs.
• A binary package for each platform that has JFFMPEG and/or fobs already packaged.
• Support for interactive media such as DVD menus and SMIL.
• Also note improvements suggested here (some of them were submitted by me).