This article is concerned with the comparison between posterior probability and p-value in two-parameter exponential distribution when the location parameter is considered as extra (nuisance) parameter. It has been shown that for a fixed p-value the posterior probability is increases as the number of observations gets large value. It means that it may be different results between classical and Bayesian point of view. This irreconcilability between classical evidence and Bayesian evidence is remained if we compare the lower bound of posterior probability under a class of reasonable prior distributions.