Accurate recognition of pig behavior plays a critical role in enhancing animal welfare and management efficiency for precision livestock farming. Nevertheless, existing methods are commonly limited by rigid feature fusion strategies, insufficient preservation of spatial details, and weak interactions across hierarchical features, which severely compromise recognition performance in complex farm scenarios. To address these challenges, we present a novel deep learning architecture termed MMBs_TransNeXt for pig behavior recognition. This model establishes a unified, hierarchical and adaptive feature learning framework. It employs Multi-stage Adaptive Feature Fusion (MAFF) to adaptively integrate semantic and fine-grained information across different network stages. It enhances the representation of subtle behavioral traits through Multi-scale Detail Fusion Convolution (MDFC). Meanwhile, it facilitates effective information communication and interaction among multi-level features via Bidirectional Modulation Feature Fusion (BMFF). Extensive experiments on a real-world pig behavior dataset validate that MMBs_TransNeXt attains state-of-the-art performance with 95.77% accuracy, substantially outperforming representative CNN and Transformer baselines. This work offers a robust technical foundation for intelligent visual monitoring and welfare evaluation in modern precision pig farming systems.