Motion analysis is essential for any video coding scheme. A moving object in a 3D environment can be analyzed better by a 3D motion model instead of 2D models, and better modeling might lead to improved coding efficiency. Gibbs formulated joint segmentation and estimation of 2D motion not only improves the performance of each stage, but also generates robust point correspondences which are necessary for rigid 3D motion estimation algorithms. Estimated rigid 3D motion parameters of a segmented object are used to find the 3D structure of those objects by minimizing another Gibbs energy. Such an approach achieves error immunity compared to linear algorithms. A more general (non-rigid) motion model can also be proposed using Gibbs formulation which permits local elastic interactions in contrast to ultimately tight rigidity between object points. Experimental results are promising for both rigid and non-rigid 3D motion models and put these models forward as strong candidates to be used in object-based coding algorithms.