Skip to content
Snippets Groups Projects

WIP: enable the VPU media driver

Closed WIP: enable the VPU media driver
martin.kepplinger/linux-next:5.7-rc4/librem5___vpu into imx8-next-librem5
2 unresolved threads
Closed Martin Kepplinger requested to merge martin.kepplinger/linux-next:5.7-rc4/librem5___vpu into imx8-next-librem5
2 unresolved threads

we should be able to test this using ffmpeg from https://gitlab.collabora.com/ezequiel/ffmpeg/tree/stateless-mpeg2-vp8-h264-v4 (preferably on the devkit over HDMI).

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
    • Guido Gunther
      Guido Gunther @guido.gunther ·
      Owner

      this should be testable without HDMI (and it would be good to do so so we can add that to pytests). Using HDMI with DCSS separate video planes is a bonus.

      EDIT: our current DCSS port is incomplete in a sense taht it likely lacking some of the necessary plane formats to support renedering directly to those. A first step would be to measure decoding performance/power usage to a buffer.

      Edited by Guido Gunther
    • Martin Kepplinger
      Martin Kepplinger @martin.kepplinger ·
      Author Owner

      or just using lcdif and verifying that the vpu is being used, right?

    • Guido Gunther
      Guido Gunther @guido.gunther ·
      Owner

      yes, but for autmoated tests buffer rendering is even nicer (for acdtual usage lcdif is the right thing (it needs to do addtional compositing due to the lock planes though)

    • Please register or sign in to reply
    • Martin Kepplinger
      Martin Kepplinger @martin.kepplinger ·
      Author Owner

      to log a bit what I do: install dependencies according to https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu

      mkdir ffmpeg_sources && cd ffmpeg_sources
git clone https://gitlab.collabora.com/ezequiel/ffmpeg.git && cd ffmpeg
git checkout stateless-mpeg2-vp8-h264-v4
PATH="$HOME/bin:$PATH" PKG_CONFIG_PATH="$HOME/ffmpeg_build/lib/pkgconfig" ./configure \
  --prefix="$HOME/ffmpeg_build" \
  --pkg-config-flags="--static" \
  --extra-cflags="-I$HOME/ffmpeg_build/include" \
  --extra-ldflags="-L$HOME/ffmpeg_build/lib" \
  --extra-libs="-lpthread -lm" \
  --bindir="$HOME/bin" \
  --enable-gpl \
  --enable-libass \
  --enable-libfreetype \
  --enable-libvorbis \
  --enable-nonfree && \
PATH="$HOME/bin:$PATH" make
    • Martin Kepplinger
      Martin Kepplinger @martin.kepplinger ·
      Author Owner

      that doesn't go well during linking:

      LD	ffmpeg_g
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_h264.o): in function `v4l2_request_h264_decode_pending_slice':
/home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_h264.c:292: undefined reference to `ff_v4l2_request_decode_slice'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_h264.o): in function `fill_dpb_entry':
/home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_h264.c:61: undefined reference to `ff_v4l2_request_get_capture_timestamp'
/usr/bin/ld: /home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_h264.c:61: undefined reference to `ff_v4l2_request_get_capture_timestamp'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_h264.o): in function `v4l2_request_h264_start_frame':
/home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_h264.c:243: undefined reference to `ff_v4l2_request_reset_frame'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_h264.o): in function `get_dpb_index':
/home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_h264.c:100: undefined reference to `ff_v4l2_request_get_capture_timestamp'
/usr/bin/ld: /home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_h264.c:100: undefined reference to `ff_v4l2_request_get_capture_timestamp'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_h264.o): in function `v4l2_request_h264_decode_slice':
/home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_h264.c:388: undefined reference to `ff_v4l2_request_append_output_buffer'
/usr/bin/ld: /home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_h264.c:383: undefined reference to `ff_v4l2_request_append_output_buffer'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_h264.o): in function `v4l2_request_h264_init':
/home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_h264.c:437: undefined reference to `ff_v4l2_request_init'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_h264.o):(.data.rel.ro+0x58): undefined reference to `ff_v4l2_request_uninit'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_h264.o):(.data.rel.ro+0x68): undefined reference to `ff_v4l2_request_frame_params'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_mpeg2.o): in function `v4l2_request_mpeg2_init':
/home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_mpeg2.c:138: undefined reference to `ff_v4l2_request_init'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_mpeg2.o): in function `v4l2_request_mpeg2_end_frame':
/home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_mpeg2.c:133: undefined reference to `ff_v4l2_request_decode_frame'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_mpeg2.o): in function `v4l2_request_mpeg2_decode_slice':
/home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_mpeg2.c:109: undefined reference to `ff_v4l2_request_append_output_buffer'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_mpeg2.o): in function `v4l2_request_mpeg2_start_frame':
/home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_mpeg2.c:80: undefined reference to `ff_v4l2_request_get_capture_timestamp'
/usr/bin/ld: /home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_mpeg2.c:83: undefined reference to `ff_v4l2_request_get_capture_timestamp'
/usr/bin/ld: /home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_mpeg2.c:102: undefined reference to `ff_v4l2_request_reset_frame'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_mpeg2.o):(.data.rel.ro+0x58): undefined reference to `ff_v4l2_request_uninit'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_mpeg2.o):(.data.rel.ro+0x68): undefined reference to `ff_v4l2_request_frame_params'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_vp8.o): in function `v4l2_request_vp8_init':
/home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_vp8.c:164: undefined reference to `ff_v4l2_request_init'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_vp8.o): in function `v4l2_request_vp8_end_frame':
/home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_vp8.c:51: undefined reference to `ff_v4l2_request_decode_frame'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_vp8.o): in function `v4l2_request_vp8_decode_slice':
/home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_vp8.c:84: undefined reference to `ff_v4l2_request_get_capture_timestamp'
/usr/bin/ld: /home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_vp8.c:86: undefined reference to `ff_v4l2_request_get_capture_timestamp'
/usr/bin/ld: /home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_vp8.c:88: undefined reference to `ff_v4l2_request_get_capture_timestamp'
/usr/bin/ld: /home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_vp8.c:159: undefined reference to `ff_v4l2_request_append_output_buffer'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_vp8.o): in function `v4l2_request_vp8_start_frame':
/home/purism/ffmpeg_sources/ffmpeg/libavcodec/v4l2_request_vp8.c:36: undefined reference to `ff_v4l2_request_reset_frame'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_vp8.o):(.data.rel.ro+0x58): undefined reference to `ff_v4l2_request_uninit'
/usr/bin/ld: libavcodec/libavcodec.a(v4l2_request_vp8.o):(.data.rel.ro+0x68): undefined reference to `ff_v4l2_request_frame_params'
collect2: error: ld returned 1 exit status
make: *** [Makefile:107: ffmpeg_g] Fehler 1
    • Please register or sign in to reply
  • Sebastian Krzyszkowiak
    Sebastian Krzyszkowiak @sebastian.krzyszkowiak ·
    Owner

    Looks like the commit that adds Hantro to imx8mq device tree is already upstream. Is there any harm in enabling the driver in our defconfig already?

  • Angus Ainslie
    Angus Ainslie @angus.ainslie ·
    Owner

    I'm ok with it as long as it doesn't increase power consumption.

  • Guido Gunther
    Guido Gunther @guido.gunther ·
    Owner

    Looks like the commit that adds Hantro to imx8mq device tree is already upstream. Is there any harm in enabling the driver in our defconfig already?

    There's currently no gstreamer that works so no need - i'm in contact with Ezequiel regarding this since some time.

    EDIT: this is due to the stateless codecs being destaged in the kernel

    Edited by Guido Gunther
  • Guido Gunther mentioned in merge request !328 (merged)

    mentioned in merge request !328 (merged)

  • Guido Gunther
    Guido Gunther @guido.gunther ·
    Owner

    Could be closed in favour of !328 (merged)

  • Martin Kepplinger
    Martin Kepplinger @martin.kepplinger ·
    Author Owner

    to log a bit what I do: install dependencies according to https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu

    mkdir ffmpeg_sources && cd ffmpeg_sources
git clone https://gitlab.collabora.com/ezequiel/ffmpeg.git && cd ffmpeg
git checkout stateless-mpeg2-vp8-h264-v4
PATH="$HOME/bin:$PATH" PKG_CONFIG_PATH="$HOME/ffmpeg_build/lib/pkgconfig" ./configure \
  --prefix="$HOME/ffmpeg_build" \
  --pkg-config-flags="--static" \
  --extra-cflags="-I$HOME/ffmpeg_build/include" \
  --extra-ldflags="-L$HOME/ffmpeg_build/lib" \
  --extra-libs="-lpthread -lm" \
  --bindir="$HOME/bin" \
  --enable-gpl \
  --enable-libass \
  --enable-libfreetype \
  --enable-libvorbis \
  --enable-nonfree && \
PATH="$HOME/bin:$PATH" make
  • Martin Kepplinger closed

    closed

Please register or sign in to reply