Browsing by Subject "Grid"
Now showing 1 - 3 of 3
Results Per Page
Sort Options
Item Open Access Parafac-spark: parallel tensor decompositions on spark(Bilkent University, 2019-08) Bekçe, Selim ErenTensors are higher order matrices, widely used in many data science applications and scienti c disciplines. The Canonical Polyadic Decomposition (also known as CPD/PARAFAC) is a widely adopted tensor factorization to discover and extract latent features of tensors usually applied via alternating squares (ALS) method. Developing e cient parallelization methods of PARAFAC on commodity clusters is important because as common tensor sizes reach billions of nonzeros, a naive implementation would require infeasibly huge intermediate memory sizes. Implementations of PARAFAC-ALS on shared and distributedmemory systems are available, but these systems require expensive cluster setups, are too low level, not compatible with modern tooling and not fault tolerant by design. Many companies and data science communities widely prefer Apache Spark, a modern distributed computing framework with in-memory caching, and Hadoop ecosystem of tools for their ease of use, compatibility, ability to run on commodity hardware and fault tolerance. We developed PARAFAC-SPARK, an e cient, parallel, open-source implementation of PARAFAC on Spark, written in Scala. It can decompose 3D tensors stored in common coordinate format in parallel with low memory footprint by partitioning them as grids and utilizing compressed sparse rows (CSR) format for e cient traversals. We followed and combined many of the algorithmic and methodological improvements of its predecessor implementations on Hadoop and distributed memory, and adapted them for Spark. During the kernel MTTKRP operation, by applying a multi-way dynamic partitioning scheme, we were also able to increase the number of reducers to be on par with the number of cores to achieve better utilization and reduced memory footprint. We ran PARAFAC-SPARK with some real world tensors and evaluated the e ectiveness of each improvement as a series of variants compared with each other, as well as with some synthetically generated tensors up to billions of rows to measure its scalability. Our fastest variant (PS-CSRSX ) is up to 67% faster than our baseline Spark implementation (PS-COO) and up to 10 times faster than the state of art Hadoop implementations.Item Open Access A prize collecting Steiner Tree approach to least cost evaluation of grid and off-grid electrification systems(Bilkent University, 2017-07) Bölükbaşı, GizemThe lack of access to electricity in developing countries necessitates spatial electricity planning for guiding sustainable electri cation projects that evaluate the costs of centralized systems vis-a-vis decentralized approaches. Heuristic approaches have been widely used in such electri cation problems to nd feasible, cost e ective solutions; however, most of the time global optimality of these solutions is not guaranteed. Our thesis through its modeling approach provides a new methodology to nd the least cost solution to this electri cation problem. We model the spatial network planning problem as Prize Collecting Steiner Tree problem which would be base for a decision support tool for rural electri cation. This new method is systematically assessed using both randomly generated data and real data from rural regions across Sub- Saharan Africa. Comparative results for the proposed approach and a widely used heuristic method are presented based on computational experiments. Additionally, a bi-objective approach that permits to take carbon emission level into the account is implemented and experimented with numerical data.Item Open Access Towards the sustainable development goals: A bi-objective framework for electricity access(Elsevier Ltd, 2021-02-01) Karsu, Özlem; Kocaman, Ayşe SelinTraditionally, the main focus of evaluation in universal electricity access problems has been cost. However, additional criteria such as increasing renewable penetration due to environmental concerns or grid penetration due to reliability concerns, have become increasingly important. We acknowledge the importance of additional criteria and propose a bi-objective framework so as to help decision makers investigate the trade-offs between potentially conflicting criteria in rural electrification. We consider two objective space based exact approaches using the Prize Collecting Steiner Tree (PCST) formulation and two metaheuristic algorithms to find Pareto solutions, and investigate their performances on real life problem instances. This study is expected to be an important decision support tool for the electrification of underdeveloped communities, having the potential of contributing to their socio-economic development.