A procedure is proposed for examining different aspects of performance for judgemental directional probability predictions of exchange rate movements. In particular, a range of new predictive performance measures is identified to highlight specific expressions of strengths and weaknesses in judgemental directional forecasts. Proposed performance qualifiers extend the existing accuracy measures, enabling detailed comparisons of probability forecasts with ex-post empirical probabilities that are derived from changes in the logarithms of the series. This provides a multi-faceted evaluation that is straightforward for practitioners to implement, while affording the flexibility of being used in situations where the time intervals between the predictions have variable lengths. The proposed procedure is illustrated via an application to a set of directional probability exchange rate forecasts for the US Dollar/Swiss Franc from 23/7/96 to 7/12/99 and the findings are discussed.