The fundamental motion model of the conventional block-based motion compensation in High Efficiency Video Coding (HEVC) is a translational motion model. However, in the real world, the motion of an object exists in the form of combining many kinds of motions. In Versatile Video Coding (VVC), a block-based 4-parameter and 6-parameter affine motion compensation (AMC) is being applied. In natural videos, in the majority of cases, a rigid object moves without any regularity rather than maintains the shape or transform with a certain rate. For this reason, the AMC still has a limit to compute complex motions. Therefore, more flexible motion model is desired for new video coding tool. In this paper, we design a perspective affine motion compensation (PAMC) method which can cope with more complex motions such as shear and shape distortion. The proposed PAMC utilizes perspective and affine motion model. The perspective motion model-based method uses four control point motion vectors (CPMVs) to give degree of freedom to all four corner vertices. Besides, the proposed algorithm is integrated into the AMC structure so that the existing affine mode and the proposed perspective mode can be executed adaptively. Because the block with the perspective motion model is a rectangle without specific feature, the proposed PAMC shows effective encoding performance for the test sequence containing irregular object distortions or dynamic rapid motions in particular. Our proposed algorithm is implemented on VTM 2.0. The experimental results show that the BD-rate reduction of the proposed technique can be achieved up to 0.45% and 0.30% on Y component for random access (RA) and low delay P (LDP) configurations, respectively.

To evaluate the performance of the proposed PAMC module, the proposed algorithm was implemented on Versatile Video Coding (VVC) Test Model 2.0 (VTM 2.0).

The VTM software uses cmake to create the needed build files.

1. Create a build directory in the root directory

     mkdir build

2. Run following cmake commands

   • Windows (sample for Visual Studio 2015 64 Bit)

       cd build
       cmake .. -G "Visual Studio 14 2015 Win64"

   • Linux Release

       cd build
       cmake .. -DCMAKE_BUILD_TYPE=Release

   • Linux Debug

       cd build
       cmake .. -DCMAKE_BUILD_TYPE=Debug

   • MACOSX Xcode

       cd build
       cmake .. -G "Xcode"

• source/App/EncoderApp

  EncAppCfg.cpp, EncAppCfg.h

• source/Lib/CommonLib

  AffineGradientSearch.cpp, AffineGradientSearch.h, CommonDef.h, Contexts.cpp, InterPrediction.cpp, InterPrediction.h, MotionInfo.h, Slice.h, TypeDef.h, Unit.cpp, Unit.h, UnitTools.cpp, UnitTools.h

• source/Lib/CommonLib/x86

  AffineGradientSearchX86.h

• source/Lib/DecoderLib

  CABACReader.cpp, DecCu.cpp, VLCReader.cpp

• source/Lib/EncoderLib

  CABACWriter.cpp, EncCfg.h, EncCu.cpp, EncCu.h, InterSearch.cpp, InterSearch.h, VLCWriter.cpp

If you want to know more about the modified source codes, please refer to the commit history.

@article{choi2019design,
  title={Design of efficient perspective affine motion estimation/compensation for versatile video coding (VVC) standard},
  author={Choi, Young-Ju and Jun, Dong-San and Cheong, Won-Sik and Kim, Byung-Gyu},
  journal={Electronics},
  volume={8},
  number={9},
  pages={993},
  year={2019},
  publisher={MDPI}
}