Video Streams

Insight3D provides a robust video reader based on RasterStream. VideoStream supports streaming from various protocols, and a large variety of media formats, codecs, frame rates and frame sizes. See below for the complete list.

A sequence of frames of a marker batch primitive using a texture created from a video stream.

Using Video Streams

Both real-time playback of video and synchronization of video with the time in Insight3D is provided by the VideoStream class. When playing in real-time, the video will play at the rate defined by the video stream, or at the frame rate specified by setting the FrameRate (get / set) property. If the Loop (get / set) property is set to true, the video will loop to the first frame when it reaches past the last frame of the video.

Note
The FrameRate (get / set) and Loop (get / set) properties are only applicable when the Playback (get / set) property is set to VideoPlayback.REAL_TIME.

Below, a VideoStream is loaded from an HTTP source, specifically a video from the AGI website.

Java

Copy

URI uri = new URI("http://www.agi.com/downloads/media-center/multimedia/current-events/us-spy-satellite/US_ASAT.wmv");
VideoStream videoData = new VideoStream(uri);
videoData.setPlayback(VideoPlayback.REAL_TIME);
videoData.setLoop(true);

The StartTime (get / set), EndTime (get / set), StartFrame (get / set), and EndFrame (get / set) properties can be used to adjust where the video starts playback and where it ends. Since these properties are interrelated, changing the StartTime (get / set) will also change the StartFrame (get / set), and vice versa. The EndTime (get / set) and EndFrame (get / set) properties work similarly.

Synchronizing Playback

When playback is set to synchronize with the SceneManager time, by setting the Playback (get / set) property to VideoPlayback.TIME_INTERVAL, the video will play synchronized with the time interval specified with the IntervalStartTime (get / set) and IntervalEndTime (get / set) properties. The start frame of the video will coincide with the start time of the interval, and the last frame of the video will coincide with the end time of that interval. For example, when Insight3D animation is played in reverse, or stepped to a specific time, the video will similarly play in reverse or step to the corresponding frame in the video at that time.

Note
For optimal synchronization with Insight3D animation time, a video format should be used that supports frame accurate seeking through indexing. Motion JPEG (mjpeg), Digital Video (dvsd), and Quicktime (mov), are some examples of codecs and formats that provide frame accurate seeking.

In the following code sample, a VideoStream is configured to play synchronized with the SceneManager time:

Java

Copy

VideoStream videoReader = new VideoStream(baseUri.resolve("ProjectedImagery/fig8.avi"));
videoReader.setPlayback(VideoPlayback.TIME_INTERVAL);
videoReader.setIntervalStartTime(getAnimation().getStartTime());
videoReader.setIntervalEndTime(new GregorianDate(2008, 5, 30, 14, 11, 58.162).toJulianDate());

A sequence of frames of a marker batch primitive using a texture created from a video stream, synchronized with the scene manager time.

As with any Raster and RasterStream, a VideoStream can be queried for its attributes and data:

Java

Copy

System.out.println(videoData.getAttributes().getWidth());
System.out.println(videoData.getAttributes().getHeight());
System.out.println(videoData.getAttributes().getFormat());
System.out.println(videoData.getAttributes().getPixelAspectRatio());

Supported Media Streams and Formats

File Extension	Description
3g2	3gp2 format
3gp	3gp format
4xm	4X Technologies format
IFF	IFF format
MTV	MTV format
RoQ	Id RoQ format
aac	ADTS AAC
ac3	raw ac3
adts	ADTS AAC
aiff	Audio IFF
alaw	pcm A law format
amr	3gpp amr file format
apc	CRYO APC format
ape	Monkey's Audio
asf	asf format
asf_stream	asf format
au	SUN AU Format
avi	avi format
avm2	Flash 9 (AVM2) format
avs	AVISynth
bethsoftvid	Bethesda Softworks 'Daggerfall' VID format
c93	Interplay C93
crc	crc testing format
daud	D-Cinema audio format
dsicin	Delphine Software International CIN format
dts	raw dts
dv	DV video format
dvd	MPEG2 PS format (DVD VOB)
dxa	dxa
ea	Electronic Arts Multimedia Format
ea_cdata	Electronic Arts cdata
ffm	ffm format
film_cpk	Sega FILM/CPK format
flac	raw flac
flic	FLI/FLC/FLX animation format
flv	flv format
framecrc	framecrc testing format
gif	GIF Animation
gxf	GXF format
h261	raw h261
h263	raw h263
h264	raw H264 video format
idcin	Id CIN format
image2	image2 sequence
image2pipe	piped image2 sequence
ingenient	Ingenient MJPEG
ipmovie	Interplay MVE format
ipod	iPod H.264 mp4 format
libnut	nut format
lmlm4	lmlm4 raw format
m4v	raw MPEG4 video format
matroska	Matroska File Format
mjpeg	MJPEG video
mm	American Laser Games MM format
mmf	mmf format
mov	mov format
mov,mp4,m4a,3gp,3g2,mj2	QuickTime/MPEG4/Motion JPEG 2000 format
mp2	MPEG audio layer 2
mp3	MPEG audio layer 3
mp4	mp4 format
mpc	musepack
mpc8	musepack8
mpeg	MPEG1 System format
mpeg1video	MPEG video
mpeg2video	MPEG2 video
mpegts	MPEG2 transport stream format
mpegtsraw	MPEG2 raw transport stream format
mpegvideo	MPEG video
mpjpeg	Mime multipart JPEG format
msnwctcp	MSN TCP Webcam stream
mulaw	pcm mu law format
mxf	MXF format
nsv	NullSoft Video format
null	null video format
nut	nut format
nuv	NuppelVideo format
ogg	Ogg format
psp	psp mp4 format
psxstr	Sony Playstation STR format
pva	pva file and stream format
rawvideo	raw video format
redir	Redirector format
rl2	rl2 format
rm	rm format
rpl	RPL/ARMovie format
rtp	RTP output format
rtsp	RTSP input format
s16be	pcm signed 16 bit big endian format
s16le	pcm signed 16 bit little endian format
s8	pcm signed 8 bit format
sdp	SDP
shn	raw shorten
siff	Beam Software SIFF
smk	Smacker Video
sol	Sierra SOL Format
svcd	MPEG2 PS format (VOB)
swf	Flash format
thp	THP
tiertexseq	Tiertex Limited SEQ format
tta	true-audio
txd	txd format
u16be	pcm unsigned 16 bit big endian format
u16le	pcm unsigned 16 bit little endian format
u8	pcm unsigned 8 bit format
vc1	raw vc1
vc1test	VC1 test bitstream format
vcd	MPEG1 System format (VCD)
vfwcap	VFW video capture
vmd	Sierra VMD format
vob	MPEG2 PS format (VOB)
voc	Creative Voice File format
wav	wav format
wc3movie	Wing Commander III movie format
wsaud	Westwood Studios audio format
wsvqa	Westwood Studios VQA format
wv	WavPack
yuv4mpegpipe	YUV4MPEG pipe format

Supported Media Codecs

4xm	8bps	8svx exponential decoder	8svx fibonacci decoder	VMware video	aasc
ac3	adpcm_4xm	adpcm_adx	adpcm_ct	adpcm_ea	adpcm_ea_r1
adpcm_ea_r2	adpcm_ea_r3	adpcm_ea_xas	adpcm_ima_amv	adpcm_ima_dk3	adpcm_ima_dk4
adpcm_ima_ea_eacs	adpcm_ima_ea_sead	adpcm_ima_qt	adpcm_ima_smjpeg	adpcm_ima_wav	adpcm_ima_ws
adpcm_ms	adpcm_sbpro_2	adpcm_sbpro_3	adpcm_sbpro_4	adpcm_swf	adpcm_thp
adpcm_xa	adpcm_yamaha	alac	amv	ape	asv1
asv2	atrac 3	avs	bethsoftvid	bmp	c93
camstudio	camtasia	cavs	cinepak	cljr	cook
cyuv	dca	dnxhd	dsicinaudio	dsicinvideo	dvbsub
dvdsub	dvvideo	dxa	escape124	ffv1	ffvhuff
flac	flashsv	flic	flv	fraps	g726
gif	h261	h263	h263i	h263p	h264
huffyuv	idcinvideo	imc	indeo2	indeo3	interplay_dpcm
interplayvideo	jpegls	kmvc	liba52	libfaac	libfaad
libgsm	libgsm_ms	libmp3lame	libtheora	libvorbis	libx264
libxvid	ljpeg	loco	mace3	mace6	mdec
mimic	mjpeg	mjpegb	mmvideo	mp2	mp3
mp3adu	mp3on4	mpc sv7	mpc sv8	mpeg1video	mpeg2video
mpeg4	mpeg4aac	mpegvideo	msmpeg4	msmpeg4v1	msmpeg4v2
msrle	msvideo1	mszh	nellymoser	nuv	pam
pbm	pcm_alaw	pcm_mulaw	pcm_s16be	pcm_s16le	pcm_s16le_planar
pcm_s24be	pcm_s24daud	pcm_s24le	pcm_s32be	pcm_s32le	pcm_s8
pcm_u16be	pcm_u16le	pcm_u24be	pcm_u24le	pcm_u32be	pcm_u32le
pcm_u8	pcm_zork	pcx	pgm	pgmyuv	png
ppm	ptx	qdm2	qdraw	qpeg	qtrle
rawvideo	real_144	real_288	rl2	roq_dpcm	roqvideo
rpza	rv10	rv20	sgi	shorten	smackaud
smackvid	smc	snow	sol_dpcm	sonic	sonicls
sp5x	sunrast	svq1	svq3	targa	theora
thp	tiertexseqvideo	tiff	truemotion1	truemotion2	truespeech
tta	txd	ultimotion	vb	vc1	vcr1
vmdaudio	vmdvideo	vorbis	vp3	vp5	vp6
vp6a	vp6f	vqavideo	wavpack	wmav1	wmav2
wmv1	wmv2	wmv3	wnv1	ws_snd1	xan_dpcm
xan_wc3	xl	xsub	zlib	zmbv

Supported Protocols

Protocol	Description	Usage
file:	Filesystem	file:///C:\video.mpg
http:	Hypertext Transfer Protocol	http://hostname/video.mpg
rtp:	Real-time Transport Protocol	rtp://hostname:port
tcp:	Transmission Control Protocol	tcp://hostname:port
udp:	User Datagram Protocol	udp://hostname:port[?option=val...] Options are as follows: 'multicast=1' : Enable multicast 'ttl=n' : Set the ttl value (for multicast only) 'localport=n' : Set the local port 'pkt_size=n' : Set the maximum packet size 'reuse=1' : Enable reusing of the socket

Supported Frame Size and Frame Rate Abbreviations

ntsc	pal	qntsc	qpal
sntsc	spal	film	ntsc-film
sqcif	qcif	cif	4cif