Flash中视频的演变（时间轴Flv → NetCon.. NetStream → Video → StageVideo →… → VideoTexture）

TKCB

网站：https://blog.gskinner.com/archiv ... video-in-flash.html
作者：gskinner


TKCB：英文原版在二楼，一楼为谷歌的中文翻译（翻译的很不错）。


视频的演变在Flash

早在好醇”几天以来，Flash在播放媒体方面非常受欢迎。在线和离线使用它来显示动画，显示演示文稿和一般广告。尽管当时没有那么多选择，但如此受欢迎仍然主要是因为它很简单。与大型笨拙的.gif相比，它能够非常平滑地为矢量艺术设置动画，并且允许用户使用简单的交互操作，例如鼠标单击和键盘输入。Macromedia Flash本身也具有非常简单的界面和UI，并且已在整个学校中用于帮助教给学生多媒体和动画。

随着互联网的发展，Flash也随之发展。随着视频播放开始流行，Flash别无选择，只能适应竞争并为用户提供最佳体验。这涉及到偷工减料，偷偷摸摸，甚至在某一时刻将一种全新的编程语言添加到组合中。

尽管进行了最大的尝试，但这并不总是那么简单。


历史

如果您记住此屏幕，请共享！

Flash动画很小，这很好，因为当时的连接速度非常慢。下载完整尺寸的.avi和.wmv视频可能要花费几个小时，并且会消耗一天中所有允许的数据使用量。但是，如今，由于.mp4和.f4v格式，连接速度提高了，视频的文件大小变得更小，并且…简而言之：Youtube。由于使用摄像机代替手动调整帧要容易得多，因此，全彩色的现实生活视频在在线上比Flash动画越来越受欢迎。

为了跟上这一趋势，Adobe在2002年还为Flash提供了在Macromedia Flash MX（6）中播放视频文件的方法。为了使播放器本身更小，Flash使用NetConnection直接从文件或URL加载了视频，然后使用NetStream对象实时解析它们。在代码中，所有要做的就是通过Actionscript中的NetConnection和NetStream加载视频，然后在屏幕上显示视频。

[Actionscript3] 纯文本查看 复制代码

var nc:NetConnection = new NetConnection();
nc.connect(null);
var ns:NetStream = new NetStream(nc);
ns.play("http://www.mySite.com/myVideo.mp4");

一如既往的简单。如此之多，以至于Youtube都使用Flash播放了很多视频。


永远不能永远解决这一问题……

但是，技术有其局限性。随着更高质量和更大屏幕的需求，视频越来越长，越来越大。在线Flash无法访问GPU，因此它依赖于CPU播放视频。尽管这样做工作了一段时间，但它仍然非常有限，并让位于许多性能问题。

想象一下，让公司的秘书CPU女士为客户的应用程序编程，而不是在另一个房间里专门为GPU的开发人员编程。当然，她可以自己做，但是她还有很多其他事情（数据组织，加载文件，打电话），并且我们不能让客户使用GPU先生，因为他是NDA，而且相当容易泄露公司机密。


艺术家对CPU看起来像数字领域的描述。

对于非常大的视频，性能成为Flash中的一个巨大问题。尤其是当人们考虑使用Flash对视频执行其他操作时，例如添加视频效果，在屏幕上添加诸如播放器UI或玩笑素材。为什么不使用Flash而不是通用播放器来播放视频呢？


嘘！

谁来处理所有这些额外的工作？女士CPU

幸运的是，Adobe并没有说“艰难”，而是无所事事。他们开始工作，并致力于使视频播放尽可能流畅。这是Flash用来播放视频的许多不同方法的简要介绍。


Video对象
播放视频的最简单答案是……实际上是Video对象。它是DisplayObject的扩展，它像任何其他Sprite一样简单地添加到舞台上。只需给视频播放NetStream，设置并忘记即可。视频播放本身由NetStream控制，因此没有其他内容。

[Actionscript3] 纯文本查看 复制代码

var video:Video = new Video();
stage.addchild(video);
video.attachNetStream(ns);

由于Video对象是一个sprite，因此可能会引起混乱。在屏幕上移动，旋转，缩放，甚至补间以在屏幕上来回反弹。


当您躺在床上而没有枕头时的播放器。

Adobe对Video对象进行了很多优化，但是它仍然完全依靠不良的CPU女士。当它也不得不面对被抛弃和转身时，滞后就会加倍。


StageVideo
某个地方的一些聪明人提出了解决GPU安全问题的方案。我们所要做的就是仅向客户端提供一组他们可以告诉GPU先生执行的命令。如果需要，可以快速讨论一下GPU。然后，所有GPU都会或可能会“回馈”直接的视频/音频回放，而无需任何额外的操作。

StageVideo出生了，男孩，这很严格。首先，您无法创建新的StageVideos；它们会自动创建为一组（台式机为4个，移动版为1个）。其次，它不是精灵，因此您不能像一个精灵一样四处移动。您所要做的就是通过设置矩形对象确定视口，应用缩放，并在应用程序允许的情况下以90度间隔旋转它。而已。

[Actionscript3] 纯文本查看 复制代码

if (stage.stageVideos) {
    var sv:StrageVideo = stage.stageVideos[0];
    sv.attachNetStream(ns);
    sv.viewBounds = new Rectangle(0, 0, 960, 480);
}

但是它播放的视频很漂亮。即使是最多的视频，也很少有性能问题。而且，由于它们全都在后台，因此开发人员仍然可以在UI之前添加UI和其他效果，以使CPU能够正常使用。他们只是无法在其背后隐藏任何东西。


闪存阶段位于STAGEVIDEO的前面。

性能得到照顾。但是，我们在此过程中失去了一件大事；自由。因为您不能将效果和功能应用于StageVideo，所以它与HTML / Javascript视频播放器在外观上没有什么不同。这还不够。设计师和用户珍惜Flash提供的自由和灵活性。

那么现在怎么办？现在变得复杂了。


你好，世界


Context3D
除了StageVideo，还创建了一个完整的框架来与GPU进行通信。这与HTML的WebGL非常相似，因此，那些了解WebGL根源的人会认识到很多方法。

首先，需要一种编程语言来使开发人员将手动命令传递给GPU。HTML使用GLSL，而Flash改编了一种称为AGAL的语言。两者在布局和功能上都非常相似，但是将它们称为两种完全不同的语言仍然是安全的。

[Actionscript3] 纯文本查看 复制代码

m44 op, va0, vc0
mov v0, va1

接下来，像HTML一样，Flash还改编了自己的Context3D对象，以获取此AGAL代码并使用GPU渲染视频。Context3D吸收了一组顶点，因此它知道要在3D空间中渲染的物理形状，并且要吸收纹理数据以在其间渲染像素。为了将其实际显示在屏幕上，Stage3D被创建为基本框架。


满足您所有3D存储需求。

所有渲染均由GPU完成。经过适当的编码和精炼，CPU几乎不需要执行任何操作，除了偶尔的数据更新之外，如果您想要的是一些有趣的事情而不是无所事事的形状。

嗯...我又在说什么？哦，是的，视频！那么视频呢？我们如何获得GPU播放视频？

可以在3D空间中渲染形状，并可以对其应用纹理。也可以动态更改纹理以创建动态图像。因此，考虑到这一点，为什么不超快速地更改一些非常大的纹理并将它们仅呈现在平坦的平面上？瞧，我们正在GPU上播放视频！

不。交换纹理需要使CPU将数据传递给GPU，这实际上需要很多时间。我的意思是说很多，特别是对于大型纹理文件。实际上，与仅使用视频对象仅通过CPU进行渲染相比，执行此类操作的性能降低。

请记住，如果您想要3D动画的精灵，做到这一点的最佳方法是将一个巨大的Spritesheet传递给GPU，并在每次更新时更改纹理的UV坐标进行播放。


编辑：谁是那个好家伙？

在您说“然后制作视频的Spritesheet！”之前，这就是行不通的原因：

让我们以24FPS 的速度拍摄2分钟的540p（960×540）视频。标准的YouTube模因视频。这是120秒乘以24帧，即2880 960×540帧。在这些帧的48×60网格中，最终得到的图像为46,080 x 32,400像素大，或1,492,992,000像素。现在尝试在Photoshop中制作一个大小的文件，然后查看它是否适合您的2GB或4GB显卡。


记住一些YouTube视频可能长达6小时。

不。最差的情况; 它会冻结屏幕，并迫使您硬重启计算机。为了防止不良程序执行此操作，上下文仅允许您传递最多2048×2048像素的纹理（为什么需要更大的纹理？），并且一次最多只能上传16个。因此它无论如何都不会成功。

那么现在怎么办？如果我们不能播放视频，为什么还要提到所有这些呢？


VideoTexture
听说过后门吗？…是的，您的房屋通常有一个，但是另一种形式的后门是代码中的开口，使人们可以有意地访问私人数据。对于客户而言，这些消息是未知的，对任何程序都是真正的威胁。程序的后门不仅可以被有恶意的人发现和利用，而且某些恶意的程序员甚至故意将后门放置在他们出于自己确切原因而提供给客户的代码中。很吓人吧？确保您可以信任您的开发人员并尊重他们。

好吧，Adobe最近为GPU创建了一个故意的后门。不用担心，这是一个很好的选择。它的名称是VideoTexture。非常友好的后门；喜欢露台上的伯爵茶和饼干。

[Actionscript3] 纯文本查看 复制代码

var ctx:Context3D = stage3D.context3D;
var vt:VideoTexture = ctx.createVideoTexture();
vt.attachNetStream(ns);

基本上，它是具有某些附加功能的单个纹理对象。您可以通过上下文获取它，类似于从舞台获取StageVideo的方法。它还引用了正在播放的NetStream，该NetStream传递了视频数据。准备就绪后，您可以像应用其他任何纹理一样将其应用于上下文。只需占用16个纹理插槽之一。没什么大不了的。然后，VideoTexture接收该视频数据，并使用其自己的后门将其直接馈送到GPU中。成功后，它将调度一个事件，让我们知道其纹理已准备就绪，然后我们可以告诉上下文以正常方式渲染。

这样，不仅将视频安全地存储在硬盘上，而且我们回到使用NetStream来控制其播放。不仅如此，它甚至还带有自己的专用update（）调用。基本上，它仅在完成下一帧的渲染时才更新，因此无需进行猜测或计时。

[Actionscript3] 纯文本查看 复制代码

vt.addEventListener(Event.TEXTURE_READY, onTextureUpdate, false, 0, true);
function onTextureUpdate(e:Event):void {
  ctx.clear(0, 0, 0, 1);
  ctx.drawTriangles(indexBuffer, 0, 2); // Just need a rectangle screen.
  ctx.present();
}

太棒了，是吗？

因此，现在我们正在使用GPU渲染视频，而这仅仅是开始。使用Starling之类的框架，或者使用AGAL手动创建程序，就可以将效果应用到视频中，例如遮罩，裁剪，滤镜，甚至是一些新的3D效果。毕竟，我们使用的是完整的3D渲染器。也可以利用其潜力。


YO DAWG，我听到了您喜欢的视频……请

注意，虽然VideoTexture在这方面仍然很新。还有一些注意事项。例如，它会根据其所处的环境自动设置自己的硬件加速状态。当它突然消失并有时几乎随机切换到软件模式时（仅使用CPU），可能导致某些不可靠的播放。另外，它仍处于beta阶段（截至目前），仅用于Adobe AIR，因此尚不能与在线Flash播放器一起使用。为此，它可能必须与WebGL配对，或者自己做一些非常疯狂的基于安全性的编码。

Web Flash Player会获得VideoTexture吗？可能不是，因为WebGL已经非常流行，并且与GPU视频回放一样容易使用。但这没关系，因为Adobe AIR更适合于制作应用而不是网页，因此在这方面仍然存在需求。


结论
即使现在，随着技术的所有进步，视频将继续以更高的帧速率，更大的分辨率和不同的文件类型进行精炼。发生这种情况时，Adobe可以选择。他们可以付出巨大的努力来跟上进度。或者，他们可以将播放功能移交给其他更离线的专业视频播放器。

无论结果如何，它们在支持和改善Flash中的视频播放方面已经走了很长一段路，而且由于CreateJS的强大功能（无意放置产品），在HTML5不断发展的世界中肯定还有一席之地和WebGL。

TKCB

web: https://blog.gskinner.com/archiv ... video-in-flash.html

The Evolution of Video in Flash

Back in the good ol’ days, Flash was very popular among many for playing media. It was used, online and offline, for displaying animations, showing presentations, and general advertising. Though there weren’t all that many options back then, such popularity still came mostly because it was simple. It was able to animate vector art very smoothly as opposed to large, clunky .gifs, as well as allow users to use simple interactions like mouse clicks and keyboard input. Macromedia Flash itself also had a very simple interface and UI, and was used in schools all over to help teach students multimedia and animation.

As the internet evolved, so did Flash. As video playback started to become popular, Flash had no choice but to adapt to keep up with competition and provide its users with the best experience possible. This involved cutting corners, taking back-doors, and even adding a whole new programming language to the mix at one point.

Despite their best attempts, it wasn’t always simple.


History

SHARE IF YOU REMEMBER THIS SCREEN!

Flash animations were considerably small, which was good because connection speeds at the time were very slow. Downloading full-sized .avi and .wmv videos could take hours and eat all the allowed data usage for the day. Nowadays however, connection speeds have increased, videos have become smaller in file-size thanks to .mp4 and .f4v formats, and… simply put: Youtube. Because using a video camera instead of manually tweaking frames was much easier, fully colored real-life videos became more and more popular online over Flash animations.

To keep up with this trend, Adobe also gave Flash the means to play video files as well in Macromedia Flash MX (6) in 2002. In order to keep the player itself small, Flash loaded videos directly from file or url using a NetConnection, and then parsed them in real-time using a NetStream object. In code, all one had to do was load the video through NetConnection and NetStream within Actionscript, and then display the video on the screen.

[Actionscript3] 纯文本查看 复制代码

var nc:NetConnection = new NetConnection();
nc.connect(null);
var ns:NetStream = new NetStream(nc);
ns.play("http://www.mySite.com/myVideo.mp4");

Simple, as always. So much so that even Youtube used Flash to play many of its videos for a while.


COULD NEVER PERMANENTLY SOLVE THIS ONE ON CHROME…

However, technology has its limits. Videos kept on getting longer and larger as higher qualities and bigger screens went into demand. Online, Flash could not access the GPU, so it was instead reliant on the CPU to play back video. Although doing this worked for a while, it was still very limited and gave way to many performance problems.

Imagine making a company’s secretary, Ms. CPU, program a client’s app instead of the specialized developer in the other room, Mr. GPU. She can do it herself, of course, but she’s got a lot of other things on her plate as well (data organization, loading files, phone calls), and we can’t let clients use Mr. GPU because he’s NDA and is quite prone to letting out company secrets.


ARTIST’S DEPICTION OF WHAT THE CPU LOOKS LIKE IN THE DIGITAL REALM.

Given a very large video, performance became a huge issue in Flash. Especially when one thinks to use Flash to do other things with the video like adding video effects, additions to the screen like player UI, or joke material. Why else use Flash for playing back video instead of a generic player if not for that?



BOING BOING!

And who handles all that extra work? Ms. CPU.

Fortunately, Adobe didn’t just say “tough luck” and sit around doing nothing. They got to work, and they committed a lot into making video playback as smooth as possible. Here’s a runthrough of the many different methods Flash used to playback video.



Video Object
The simplest answer to playing video was… actually, IS the Video object. It’s an extension of a DisplayObject, which is simply added to the stage like any other sprite. Simply give the Video the playing NetStream, set, and forget. The video playback itself is controlled by the NetStream, so there’s nothing else to it.

[Actionscript3] 纯文本查看 复制代码

var video:Video = new Video();
stage.addchild(video);
video.attachNetStream(ns);

Since the Video object is a sprite, it could be messed with. Moved around the screen, rotated, scaled, even tweened to bounce back and forth around the screen.


A PLAYER FOR WHEN YOU’RE LYING IN BED WITHOUT A PILLOW.

Adobe put a lot of optimizations into the Video object, but it was still fully reliant on poor Ms. CPU. When it also had to deal with being tossed and turned around, the lag multiplied.


StageVideo
Some brilliant mind somewhere came up with a solution to the GPU security problem. All we had to do was only give the client a predetermined set of commands they can tell Mr. GPU to do. A quick-chat for the GPU, if you will. Then all the GPU would, or rather “could”, give back is direct video/audio playback without anything extra.

StageVideo was born, and boy, is it strict. First, you cannot create new StageVideos; they are automatically created in a set of 4 for desktop and 1 for mobile. Second, it’s not a sprite, so you cannot move it around like one. All you can do is determine the viewport by setting a rectangle object, apply a zoom, and rotate it in 90 degree intervals if given permission by the app. That’s it.

[Actionscript3] 纯文本查看 复制代码

if (stage.stageVideos) {
    var sv:StrageVideo = stage.stageVideos[0];
    sv.attachNetStream(ns);
    sv.viewBounds = new Rectangle(0, 0, 960, 480);
}

But it plays videos beautifully. Very few performance issues even with the largest of videos. And since it’s all in the background, the developer can still add a UI and additional effects in front of it for the CPU to mess with. They just cannot hide anything behind it.



THE FLASH STAGE IS IN FRONT OF STAGEVIDEO.

Performance was taken care of. However, we lost one big thing in the process; freedom. Because you cannot apply effects and features to StageVideo, it became no different (visually) from an HTML/Javascript video player. This wasn’t going to be enough; designers and users cherished the freedom and flexibility that Flash provided.

So now what? Now it gets complicated.



HELLO WORLD


Context3D
Alongside StageVideo, an entire framework was created to communicate with the GPU. This is very similar to HTML’s WebGL, so those of you who know the roots of WebGL will recognize a lot of the methodology.

First, a programming language was needed to let the developer pass manual commands to the GPU. While HTML uses GLSL, Flash adapted a language called AGAL. Both are very similar in layout and functionality, but it’s still safe to call them two fairly different languages.

[Actionscript3] 纯文本查看 复制代码

m44 op, va0, vc0
mov v0, va1

Next, like HTML, Flash also adapted its own Context3D object to take this AGAL code and render video with it using the GPU. The Context3D takes in a set of vertex points so it knows what physical shapes to render in 3D space, and it takes in texture data to render the pixels in between. To physically display it on the screen, Stage3D was created as a base framework.


FOR ALL YOUR 3D STORAGE NEEDS.

All rendering is done by the GPU. Properly coded and refined, the CPU barely has to do anything at all beyond the occasional data update if you want something more interesting than a bunch of shapes hanging around doing nothing.

Um… what was I talking about again? Oh yeah, video! So what about video? How do we get the GPU to play video?

Shapes can be rendered in 3D space and textures can be applied to them. Textures can also be changed on the fly to create dynamic images as well. So, with that in mind, why not change some very large textures super-quickly and render those just on a flat plain? Lo and behold, we’re playing video on the GPU!

Nope. Swapping out textures involves making CPU pass the data to the GPU, which actually takes a lot of time. I mean a LOT, especially for large texture files. In fact, it’s way less performant to do something like this than to simply use Video Object to render with just the CPU.

That in mind, if you wanted an animated sprite in 3D, the best way to do this is to pass the GPU a giant spritesheet and play that by changing the texture’s UV coordinates with each update.



EDIT: WHO’S THAT HANDSOME FELLA?

Before you say “then make a spritesheet of the video!”, here’s why that won’t work:

Let’s take a 2-minute 540p (960×540) video at 24FPS. Standard Youtube meme video. That’s 120 seconds times 24 frames, which is 2880 960×540 frames. In a 48×60 grid of these frames, we end up with an image that’s 46,080 x 32,400 pixels large, or 1,492,992,000 pixels. Now try to make a file that size in Photoshop and see if that will fit in your 2GB or 4GB graphics card.



KEEP IN MIND SOME YOUTUBE VIDEOS CAN GO UP TO 6 HOURS.

Nope. Worst case; it will freeze your screen and force you to hard-restart your computer. To prevent bad programs from doing this, the context will only let you pass in textures up to 2048×2048 pixels (why would you need any larger?), and you can only upload up to 16 of them at a time. So it won’t work out anyways.

So… now what? Why even mention all this if we can’t play videos?



VideoTexture
Heard of back-doors before? … yes, your house usually has one, but another form of back-doors are openings in code where somebody can access otherwise intentionally private data. Unannounced and unknown to the client, these are a real threat to any and all programs. Not only could a program’s back-door be found and exploited by someone with malicious intent, but some malicious programmers even purposely put backdoors in their own code that they give to their clients for that exact reason. Pretty scary, eh? Make sure you can trust your developers and treat them with respect.

Well, Adobe has recently created an intentional back-door for the GPU. Don’t worry, it’s a good one. It’s name is VideoTexture. Very friendly back-door; likes earl grey tea and biscuits on the patio.

[Actionscript3] 纯文本查看 复制代码

var ctx:Context3D = stage3D.context3D;
var vt:VideoTexture = ctx.createVideoTexture();
vt.attachNetStream(ns);

Basically, it’s a single texture object with some extra functionality. You get it through the context, similar to how you get a StageVideo from the stage. It also takes in a reference to a playing NetStream, which passes in video data. When it’s ready to go, you apply to the context like any other texture. Just takes up one of the sixteen texture slots. No big deal. VideoTexture then takes that video data and, using its own self-made backdoor, directly feeds it right into the GPU. When successful, it dispatches an event that lets us know that its texture is ready, and then we can tell the context to render as normal.

This way, not only is the video stored safely on the hard drive, but we’re back to using NetStream to control its playback. Not just that, but it even comes with its own specialized update() call. Basically, it only updates when it’s finished rendering the next frame, so there’s no guesswork or timing involved.

[Actionscript3] 纯文本查看 复制代码

vt.addEventListener(Event.TEXTURE_READY, onTextureUpdate, false, 0, true);
function onTextureUpdate(e:Event):void {
  ctx.clear(0, 0, 0, 1);
  ctx.drawTriangles(indexBuffer, 0, 2); // Just need a rectangle screen.
  ctx.present();
}

Pretty awesome, eh?

So now we’re using the GPU to render the video, and that’s only the start. Using frameworks like Starling, or manually creating programs using AGAL, one can apply effects to the video like masking, cropping, filters, and even some new 3D effects. After all, we are using a full 3D renderer to do this. May as well take advantage of its potential.



YO DAWG, I HEARD YOU LIKED VIDEOS…

Mind though that VideoTexture is still new at this. There are still some limitations to watch out for. For example, it automatically sets its own hardware acceleration status based on what it thinks its environment is like. This can lead to some unreliable playback when it freaks out and switches to software mode almost randomly at times (only uses CPU). Also, it’s still in a beta stage (as of right now), and is only used for Adobe AIR, so it cannot yet be used with online Flash players. For that to happen, it may have to either pair up with WebGL, or do some pretty crazy security-based coding on their own.

Will the web Flash Player ever get VideoTexture? Probably not, since WebGL has already taken off quite spectacularly and is just as easy to work with for GPU video playback. But that’s OK, since Adobe AIR is more for making apps instead of web pages, there still is a demand in that regard.



Conclusion
Even now, with all the advancements in technology, videos will continue to get refined with higher framerates, larger resolutions, and different file types. When that happens, Adobe has a choice. They can either put in the vast amounts of effort required to keep up with the advancements. Or, they can surrender playback functionality to the other, more specialized video players on and offline.

Regardless of the outcome, they have already come a long ways in supporting and improving video playback in Flash, and thanks to the awesome power of CreateJS (no product placement intended), there’s definitely still a place for them in the widely evolving world of HTML5 and WebGL.

TKCB

非常不错的文字，顶！！

huwy

大赞！写的非常好，棒棒哒！

river

最近正好在看视频相关的 找机会用用videotexture qvq

danceman

大赞！写的非常好，棒棒哒！

lazy_yu

as3 永不寂寞。即使web没有一席之地

guanguan

flash5的UI图标还是像素的

wuliao2014

不错的方案，值得研究，看能不能支持一万左右的分辨率播放。