ExoPlayer stuck in buffering after re-adding the surface view a few time

I took a look at this and was able to reproduce, thanks. It’s a fundamental limitation of MediaCodec (in the Android platform) that it needs a Surface attached to it at all times. When the SurfaceView is removed from the view the underlying Surface is destroyed. This in turn forces ExoPlayer to destroy its MediaCodec instance, because there’s no longer a Surface for it to be attached to.

When the SurfaceView is added back to the view hierarchy its underlying Surface is created again. ExoPlayer creates a new MediaCodec instance, however since it’s a new instance it can only start decoding from the next key-frame. It does this and displays the next key-frame immediately, then waits for the playback position to reach the position of that key-frame before rendering subsequent frames. This is why even after a single remove/add cycle, you observe that the first frame that’s displayed is from the future and frozen for a while.

If you remove/add multiple times the player skips ahead 1 key-frame each time. If you do this enough, the player gets stuck in a weird state where it’s buffered a long way into the future compared to the correct playback position, but by successively rendering all of the key-frames it actually doesn’t have any frame to render. The player doesn’t think it needs to buffer, but it’s also in a state where it cannot transition to the playing state because it hasn’t rendered a frame.

The fundamental limitation of MediaCodec is a real pain, unfortunately. There are however things we can do in the library, and things you can do in your application:

On the library side:

• We should delay rendering the next key-frame until the playback position actually reaches the corresponding time. This will change the behavior from seeing that frame frozen for a while to seeing a blank surface for a while. Neither is a good user experience, but the latter has the benefit that the video renderer is prevented from skipping arbitrarily far ahead as a result of multiple remove/add cycles. This will prevent the stuck buffering state and allow playback to always continue properly from the next key-frame regardless of how many remove/add cycles occur, so will in general produce more predictable behavior.

On the application side, there are a bunch of things you can do to try and provide a better user experience.

• On API level 23 and above you can solve this problem using DummySurface, which we recently added to the library. This gives proper seamless re-join. It wont work prior to API level 23, because the approach relies on MediaCodec.setOutputSurface. The basic idea is to use the DummySurface as a means of ensuring there’s always a surface attached to the MediaCodec. Rather than calling player.setVideoSurfaceView, manage the Surface lifecycle yourself. Something like this should work well from your activity:

    SurfaceManager manager = new SurfaceManager(player);
    surface.getHolder().addCallback(manager);

With:

 private static final class SurfaceManager implements SurfaceHolder.Callback {

    private final SimpleExoPlayer player;
    private DummySurface dummySurface;

    public SurfaceManager(SimpleExoPlayer player) {
      this.player = player;
    }

    @Override
    public void surfaceCreated(SurfaceHolder holder) {
      player.setVideoSurface(holder.getSurface());
    }

    @Override
    public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
      // Do nothing.
    }

    @Override
    public void surfaceDestroyed(SurfaceHolder holder) {
      if (dummySurface == null) {
        dummySurface = DummySurface.newInstanceV17(false);
      }
      player.setVideoSurface(dummySurface);
    }

    public void release() {
      if (dummySurface != null) {
        dummySurface.release();
        dummySurface = null;
      }
    }

  }

In the future, we hope to be able to hook DummySurface up automatically inside of the player for cases where it helps.

• For pre-API-23 there are complicated solutions involving off-screen rendering into GL textures. It would take quite a bit of time and effort to put together sample code, however.

• One easier option is to disable and re-enable the video renderer, but this causes re-buffering and so isn’t ideal. Nevertheless, you can do it something like this:

  private static final class SurfaceManager implements SurfaceHolder.Callback {

    private final SimpleExoPlayer player;
    private final DefaultTrackSelector trackSelector;

    public SurfaceManager(SimpleExoPlayer player, DefaultTrackSelector trackSelector) {
      this.player = player;
      this.trackSelector = trackSelector;
    }

    @Override
    public void surfaceCreated(SurfaceHolder holder) {
      player.setVideoSurface(holder.getSurface());
      trackSelector.setRendererDisabled(VIDEO_RENDERER_INDEX, false);
    }

    @Override
    public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
      // Do nothing.
    }

    @Override
    public void surfaceDestroyed(SurfaceHolder holder) {
      player.setVideoSurface(null);
      trackSelector.setRendererDisabled(VIDEO_RENDERER_INDEX, true);
    }

  }

Thanks. I cannot reproduce the issue with the latest dev-v2. Could you give that a try to see if you see the same behavior there? I just pushed some recent changes to dev-v2 to make sure you have all of the same changes I tested with, so be sure to pull them if you give it a try.