We propose a new Bayesian classifier, based on the recently introduced causal Markov random field (MRF) model, Quadrilateral MRF (QMRF). We use a second order inhomogeneous anisotropic QMRF to model the prior and likelihood probabilities in the maximum a posteriori (MAP) classifier, named here as MAP-QMRF. The joint distribution of QMRF is given in terms of the product of two dimensional clique distributions existing in its neighboring structure. 20 manually labeled human brain MR images are used to train and assess the MAP-QMRF classifier using the jackknife validation method. Comparing the results of the proposed classifier and FreeSurfer on the Dice overlap measure shows an average gain of 1.8%. We have performed a power analysis to demonstrate that this increase in segmentation accuracy substantially reduces the number of samples required to detect a 5% change in volume of a brain region.