Computer technology has long been applied to language testing as a time and cost efficient way to conveniently assess the proficiency of large numbers of students. Thus, a good deal of research have focused on the effect and efficiency of computer assisted (semi-direct) assessment in evaluating different constructs of the language. Nonetheless, little research has been conducted to compare computer assisted and face-to-face (direct) formats to find whether the two modes yield similar results in oral assessment and whether one is advantageous over the other. Even less investigated were the possible outcomes of administration of computer-assessited speaking tests on a local basis, as achievement tests. The purpose of this exploratory study is to fill the abovementioned gap via examining the relationships between a number of variables. Presented in the thesis are the relationships between test scores obtained in two different test modes at two different proficiency levels, the students’ perceptions of the test modes, and their anxiety levels with regard to speaking in a foreign language, speaking tests, and using computers. Data were collected through four computer assisted and four faceto-face speaking assessments, a questionnaire on Computer Asssisted Speaking Assesment (CASA) perceptions and another on Face-to-face Speaking Assessment (FTFsa) perceptions, a speaking test and speaking anxiety questionnaire, and a computer familiarity questionnaire. A total of 66 learners of English at tertiary level and four instructors of English participated in the study which was conducted at Uludağ University School of Foreign Languages. The quantitative and qualitative data analyses revealed that the two test modes give very different rankings to the students, and the students’ perceptions of the test modes, which have been found to be more positive about the FTFsa at both proficiency levels, are not strongly related to their performance in the speaking tests. The relationship between different types of anxiety mentioned above and test scores are only weakly related to the test scores and the degree of the relationships vary depending on the proficiency level. The results of this study are hoped to be beneficial to the language assessors, instructors, and institutions and researchers that are into language assessment.