Browsing by Subject "Calibration"
Now showing 1 - 12 of 12
Results Per Page
Sort Options
Item Open Access “Because she is a know-it-all”: school-aged children’s understanding of calibration for hesitant informants(Bilkent University, 2023-07) Sunay, OnurCalibration refers to the extent to which one’s confidence predicts their accuracy. Accordingly; someone accurate and confident, and someone inaccurate and hesitant are well-calibrated; and someone inaccurate and confident, and someone accurate but hesitant are poorly calibrated. Although there is evidence of adults’ calibration understanding, children do not have a complete understanding of calibration. The current study aimed to investigate children’s calibration understanding better. To that end, 7-, 9-, and 11-year-old children were tested on three calibration tasks with informants that included the inaccurate and hesitant informant. The tasks included explicit and implicit measures of calibration. The results showed that children performed similarly across all ages, but there were differences in how children performed between different tasks. Also, accuracy had more influence on children’s judgments for who was a reliable informant than confidence. Third, more children passed the implicit calibration task but failed the explicit one than vice versa. Lastly, children’s calibration understanding was not related to their executive function (EF) abilities. These results suggest that calibration is a complex ability influenced by social situations. The role situations play and how they might be used as a broader framework to explain calibration are highlighted in the discussion. EF and other cognitive abilities that might be related to calibration understanding are also discussed.Item Open Access The effects of feedback and training on the performance of probability forecasters(Elsevier, 1992) Benson, P. G.; Önkal D.An experiment examined the effects of outcome feedback and three types of performance feedback-calibration feedback, resolution feedback, and covariance feedback - on various aspects of the performance of probability forecasters. Subjects made 55 forecasts in each of four sessions, receiving feedback prior to making their forecasts in each of the last three sessions. The provision of calibration feedback was effective in improving both the calibration and overforecasting of probability forecasters, but the improvement was not gradual; it occurred in one step, between the second and third sessions. Simple outcome feedback had very little effect on forecasting performance. Neither resolution nor covariance feedback affected forecasters' performances much differently than outcome feedback. However, unlike outcome feedback, the provision of performance feedback caused subjects to manage their use of the probability scale. Subjects switched from two-digit probabilities to one-digit probabilities, and those receiving calibration and resolution feedback also reduced the number of different probabilities they used. © 1992.Item Open Access The effects of feedback on judgmental interval predictions(Elsevier, 2004) Bolger, F.; Önkal-Atay, D.The majority of studies of probability judgment have found that judgments tend to be overconfident and that the degree of overconfidence is greater the more difficult the task. Further, these effects have been resistant to attempts to ‘debias’ via feedback. We propose that under favourable conditions, provision of appropriate feedback should lead to significant improvements in calibration, and the current study aims to demonstrate this effect. To this end, participants first specified ranges within which the true values of time series would fall with a given probability. After receiving feedback, forecasters constructed intervals for new series, changing their probability values if desired. The series varied systematically in terms of their characteristics including amount of noise, presentation scale, and existence of trend. Results show that forecasts were initially overconfident but improved significantly after feedback. Further, this improvement was not simply due to ‘hedging’, i.e. shifting to very high probability estimates and extremely wide intervals; rather, it seems that calibration improvement was chiefly obtained by forecasters learning to evaluate the extent of the noise in the series. D 2003 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved.Item Open Access Effects of feedback on probabilistic forecasts of stock prices(1995) Önkal D.; Muradoğlu, G.This paper reports the results of an experiment in stock-price forecasting that investigated the effects of feedback on various dimensions of probability forecasting accuracy. Three types of feedback were used: (1) simple outcome feedback, (2) outcome feedback presented in the task format, and (3) performance feedback in the form of an overall accuracy score in addition to detailed calibration information. While calibration improved for all the feedback groups, forecasters' skill was found to improve only for the task-formated outcome feedback and performance feedback groups (but not for the simple outcome feedback group). Finally, the forecasters in the performance feedback group also improved their mean slope and mean probability scores, an effect not observed in the other feedback groups. It is suggested that, in a dynamic environment like the stock market, probability forecasting offers distinct advantages by providing an important channel of communication between the forecasters and the users of financial information.Item Open Access An exploratory analysis of portfolio managers' probabilistic forecasts of stock prices(John Wiley & Sons, 1994) Önkal D.; Muradoğlu, G.This study reports the results of an experiment that examines (1) the effects of forecast horizon on the performance of probability forecasters, and (2) the alleged existence of an inverse expertise effect, i.e., an inverse relationship between expertise and probabilistic forecasting performance. Portfolio managers are used as forecasters with substantive expertise. Performance of this ‘expert’ group is compared to the performance of a ‘semi‐expert’ group composed of other banking professionals trained in portfolio management. It is found that while both groups attain their best discrimination performances in the four‐week forecast horizon, they show their worst calibration and skill performances in the 12‐week forecast horizon. Also, while experts perform better in all performance measures for the one‐week horizon, semi‐experts achieve better calibration for the four‐week horizon. It is concluded that these results may signal the existence of an inverse expertise effect that is contingent on the selected forecast horizon.Item Open Access Fast system calibration with coded calibration scenes for magnetic particle imaging(IEEE, 2019) İlbey, Serhat; Top, C. B.; Güngör, Alper; Çukur, Tolga; Sarıtaş, Emine Ülkü; Güven, H. EmreMagnetic particle imaging (MPI) is a relatively new medical imaging modality, which detects the nonlinear response of magnetic nanoparticles (MNPs) that are exposed to external magnetic fields. The system matrix (SM) method for MPI image reconstruction requires a time consuming system calibration scan prior to image acquisition, where a single MNP sample is measured at each voxel position in the field-of-view (FOV). The scanned sample has the maximum size of a voxel so that the calibration measurements have relatively poor signal-to-noise ratio (SNR). In this paper, we present the coded calibration scene (CCS) framework, where we place multiple MNP samples inside the FOV in a random or pseudo-random fashion. Taking advantage of the sparsity of the SM, we reconstruct the SM by solving a convex optimization problem with alternating direction method of multipliers using CCS measurements. We analyze the effects of filling rate, number of measurements, and SNR on the SM reconstruction using simulations and demonstrate different implementations of CCS for practical realization. We also compare the imaging performance of the proposed framework with that of a standard compressed sensing SM reconstruction that utilizes a subset of calibration measurements from a single MNP sample. The results show that CCS significantly reduces calibration time while increasing both the SM reconstruction and image reconstruction performances.Item Open Access Feedback-labelling synergies in judgmental stock price forecasting(Elsevier, 2004) Goodwin, P.; Önkal-Atay, D.; Thomson, M. E.; Pollock, A. C.; Macaulay, A.Research has suggested that outcome feedback is less effective than other forms of feedback in promoting learning by users of decision support systems. However, if circumstances can be identified where the effectiveness of outcome feedback can be improved, this offers considerable advantages, given its lower computational demands, ease of understanding and immediacy. An experiment in stock price forecasting was used to compare the effectiveness of outcome and performance feedback: (i) when different forms of probability forecast were required, and (ii) with and without the presence of contextual information provided as labels. For interval forecasts, the effectiveness of outcome feedback came close to that of performance feedback, as long as labels were provided. For directional probability forecasts, outcome feedback was not effective, even if labels were supplied. Implications are discussed and future research directions are suggested.Item Open Access Improved deterministic measurement model for consumer-grade accelerometers(Institution of Engineering and Technology, 2016) Barshan, B.; Seçer, G.Deterministic error modelling, calibration and model parameter estimation of consumer-grade accelerometers is considered and improvement to the traditionally used measurement model is proposed. Calibration experiments on a flight motion simulator are performed for experimental verification. Model parameters are estimated using the Levenberg-Marquardt optimisation algorithm. Residual errors are considerably reduced as a result of the improved measurement model.Item Open Access Learning-based reconstruction methods for magnetic particle imaging(Bilkent University, 2023-01) Güngör, AlperMagnetic particle imaging (MPI) is a novel modality for imaging of magnetic nanoparticles (MNP) with high resolution, contrast and frame rate. An inverse problem is usually cased for reconstruction, which requires a time-consuming calibration scan for measuring a system matrix (SM). Previous calibration procedures involve scanning an MNP filled sample with a size that matches desired resolution through field of view. This time-consuming calibration scan which accounts for both system and MNP response imperfections is a critical factor prohibiting its practical use. Moreover, the quality of the reconstructed images heavily depend on the prior information about the MNP distribution as well as the specific re-construction algorithm, since the inverse problem is highly ill-posed. Previous approaches commonly solve an optimization problem based on the measurement model that iteratively estimates the image while enforcing data consistency in an interleaved fashion. However, while conventional hand-crated priors do not fully capture the underlying complex features of MPI images, recently proposed learned priors suffer from limited generalization performance. To tackle these issues, we first propose a deep learning based technique for accelerated MPI calibration. The technique utilizes transformers for SM super-resolution (TranSMS) for accelerated calibration of SMs with high signal-to-noise-ratio. For signal-to-noise-ratio efficiency, we propose scanning a low resolution SM with larger MNP sample size. For improved SM estimation, TranSMS leverages the vision trans-former to capture global contextual information while utilizing the convolutional module for local high-resolution features. Finally, a novel data-consistency module enforces measurement fidelity. TranSMS is shown to outperform competing methods significantly in terms of both SM recovery and image reconstruction performance. Next, to improve image reconstruction quality, we propose a novel physics-driven deep equilibrium based technique with learned consistency block for MPI (DEQ-MPI). DEQ-MPI embeds deep network operators into iterative optimization procedures for improved modeling of image statistics. Moreover, DEQ-MPI utilizes learned consistency to better capture the data statistics which helps improve the overall image reconstruction performance. Finally, compared to previous unrolling-based techniques, DEQ-MPI leverages implicit layers which enables training on the converged output. Demonstrations on both simulated and experimental data show that DEQ-MPI significantly improves image quality and reconstruction time over state-of-the-art reconstructions based on hand-crafted or learned priors.Item Open Access Manyetik parçacık görüntüleme için evrişimsel sinir ağı tabanlı bir süper-çözünürlük tekniği(IEEE, 2021-07-19) Aşkın, Barış; Güngör, Alper; Soydan, Damla Alptekin; Top, Can Barış; Çukur, TolgaManyetik Parçacık Görüntüleme (MPG), süperparamanyetik demir-oksit (SPDO) parçacıklarının yüksek çözünürlük ve kare hızında görüntülenmesini sağlayan bir görüntüleme yöntemidir. Görüntüleme işlemi doğrusal olarak modellenebilmektedir. Ancak deneysel sistemlerin ideal dışı davranışı ve teorik sistemlere kıyasla değişimlerinden dolayı, MPG sistemlerinde çoğu durumda öncelikli olarak ileri model matrisi ölçülür (sistem kalibre edilir) ve ardından bu matrisler kullanılarak görüntülerin geriçatımı yapılır. Görüntü çözünürlüğü ve boyutu doğrudan sistem matrisinin boyutundan etkilenmektedir. Ancak, kalibrasyon işlemi görüntüleme alanına bağlı olarak çok zaman almaktadır. Bu çalışmada, düşük çözünürlükte ölçülen sistem matrisleri üzerinde süper-çözünürlük teknikleri kullanılarak yüksek çözünürlüklü sistem matrisi elde edilmesi önerilmektedir. Bu amaç doğrultusunda evrişimsel sinir ağı (ESA) tabanlı bir süperçözünürlük tekniği MPG için uyarlanmış ve doğrusal aradeğerlemeye (interpolasyon) karşı etkinliği gösterilmiştir. Yöntemler gürültüsüz bir benzetim ortamında kıyaslanmış ve 4 4 kat süper-çözünürlük için, önerilen yöntem %2.92 normalize edilmiş ortalama kare hatasına yol açarken, bikübik aradeğerlemenin %12.47 hataya yol açtığı gösterilmiştir.Item Open Access Projection onto epigraph sets for rapid self-tuning compressed sensing MRI(IEEE, 2019) Shahdloo, Mohammad; Ilıcak, Efe; Tofighi, Mohammad; Sarıtaş, Emine Ülkü; Çetin, A. Enis; Çukur, TolgaThe compressed sensing (CS) framework leverages the sparsity of MR images to reconstruct from undersampled acquisitions. CS reconstructions involve one or more regularization parameters that weigh sparsity in transform domains against fidelity to acquired data. While parameter selection is critical for reconstruction quality, the optimal parameters are subject and dataset specific. Thus, commonly practiced heuristic parameter selection generalizes poorly to independent datasets. Recent studies have proposed to tune parameters by estimating the risk of removing significant image coefficients. Line searches are performed across the parameter space to identify the parameter value that minimizes this risk. Although effective, these line searches yield prolonged reconstruction times. Here, we propose a new self-tuning CS method that uses computationally efficient projections onto epigraph sets of the ℓ1 and total-variation norms to simultaneously achieve parameter selection and regularization. In vivo demonstrations are provided for balanced steady-state free precession, time-of-flight, and T1-weighted imaging. The proposed method achieves an order of magnitude improvement in computational efficiency over line-search methods while maintaining near-optimal parameter selection.Item Open Access TranSMS: transformers for super-resolution calibration in magnetic particle imaging(Institute of Electrical and Electronics Engineers Inc., 2022-07-11) Gungor, Alper; Askin, Baris; Soydan, D.A.; Saritas, Emine Ulku; Top, C. B.; Çukur, TolgaMagnetic particle imaging (MPI) offers exceptional contrast for magnetic nanoparticles (MNP) at high spatio-temporal resolution. A common procedure in MPI starts with a calibration scan to measure the system matrix (SM), which is then used to set up an inverse problem to reconstruct images of the MNP distribution during subsequent scans. This calibration enables the reconstruction to sensitively account for various system imperfections. Yet time-consuming SM measurements have to be repeated under notable changes in system properties. Here, we introduce a novel deep learning approach for accelerated MPI calibration based on Transformers for SM super-resolution (TranSMS). Low-resolution SM measurements are performed using large MNP samples for improved signal-to-noise ratio efficiency, and the high-resolution SM is super-resolved via model-based deep learning. TranSMS leverages a vision transformer module to capture contextual relationships in low-resolution input images, a dense convolutional module for localizing high-resolution image features, and a data-consistency module to ensure measurement fidelity. Demonstrations on simulated and experimental data indicate that TranSMS significantly improves SM recovery and MPI reconstruction for up to 64-fold acceleration in two-dimensional imaging