Scalable Analysis of Open Data Graphs

机译：开放数据图的可扩展分析

获取原文

页面导航

摘要
著录项
相似文献
相关主题

摘要

We have studied Open Data as a connected graph. Each data package is considered a vertex, and we studied the similarity graph induced by several different similarity measures. We analyzed the resulting similarity graph using different metrics to estimate its quality and informativeness. In order to cope with the size of the open data graph (over 6 billion edges), the graph constructions and analysis are done using a distributed computation framework, Apache Spark. The algorithms were implemented using the Spark resilient distributed data algebra, and executed on the Google Cloud Platform (GCP).

机译：我们已经研究了开放数据作为连接图。每个数据包都被视为一个顶点，并且我们研究了由几种不同的相似性度量得出的相似性图。我们使用不同的指标分析了所得相似度图，以评估其质量和信息性。为了应付开放数据图的大小（超过60亿条边），使用分布式计算框架Apache Spark完成图的构建和分析。该算法使用Spark弹性分布式数据代数实现，并在Google Cloud Platform（GCP）上执行。

著录项

来源
《International Conference on Information Reuse and Integration for Data Science》|2019年|334-341|共8页
会议地点 Los Angeles(US)
作者
Andrei Stoica; Michael Valdron; Ken Pu;
展开▼
作者单位

UOIT Canada;

展开▼
会议组织
原文格式 PDF
正文语种
中图分类
关键词
Metadata; Distributed databases; Measurement; Cluster computing; Government; Correlation; Sparks;

机译：元数据分布式数据库；测量;集群计算；政府;相关性火花;

相似文献

外文文献
中文文献
专利

1. Short-term sediment dynamics on a meso-scale contourite drift (off NW Iberia): Impacts of multi-scale oceanographic processes deduced from the analysis of mooring data and numerical modelling [J] . Zhang Wenyan, Hanebuth Till J. J., Stoeber Uwe Marine Geology . 2016,第8期

机译：中尺度等高线漂移（近西北伊比利亚）上的短期沉积物动力学：从系泊数据分析和数值模型推论的多尺度海洋学过程的影响
2. Applications of Data Assimilation to Analysis of the Ocean on Large Scales (gtSpecial IssueltData Assimilation in Meteology and Oceanography: Theory and Practice) [J] . Antonio J. Busalacchi, Eric C. Hackert, Robert N. Miller ournal of the Meteorological Society of Japan . 1997,第1B期

机译：数据同化在大规模海洋分析中的应用（gtSpecial Issuelt气象和海洋学中的数据同化：理论与实践）
3. Topological Patterns for Scalable Representation and Analysis of Dataflow Graphs [J] . Nimish Sane, Hojin KeeGunasekaran Seetharaman, Shuvra S. Bhattacharyya Journal of VLSI signal processing systems . 2011,第2期

机译：用于数据流图的可扩展表示和分析的拓扑模式
4. Centrality and Scalability Analysis on Distributed Graph of Large-Scale E-mail Dataset for Digital Forensics [C] . Selim Ozcan, Merve Astekin, Narasimha K. Shashidhar, IEEE International Conference on Big Data . 2020

机译：用于数字取证的大型电子邮件数据集分布图的中心性和可扩展性分析
5. Scalable Machine Learning Using Dataflow Graph Analysis [D] . ?Huang, Chien-Chin 2019

机译：使用DataFlow Graph分析的可扩展机器学习
6. CHICKN: extraction of peptide chromatographic elution profiles from large scale mass spectrometry data by means of Wasserstein compressive hierarchical cluster analysis [O] . Olga Permiakova, Romain Guibert, Alexandra Kraut, 2021

机译：鸡：通过Wassersein压缩等级簇分析从大规模质谱数据提取肽色谱摘谱曲线
7. Decision-making is the process of analyzing information about a problem situation and comparing it to a specific conclusion in order to solve a specific problematic (Yıkılmaz, 2001; Miller and Byrnes, 2001). Decision-making styles are a mechanism that is influenced by the internal and external conditions that determine the direction of the decisions of the individual, the content of the decision-making process, and the outcome of the decision-making process (Payne, Bettman and Johson, 1993; Bavol’ár and Orosová, 2015). ACT is a contemporary member of the Cognitive Behavioral Therapy family. ACT (Acceptance and commitment therapy) has both similar and different directions with Behavioral Therapies and Cognitive Therapies (Herbet and Forman, 2011; Hayes, 2004). KKT responds to classical behavioral treatments using both existential and cognitive approaches in the analysis of behavior. KKT is a science wing that tries to solve human problems with a wider perspective aimed at solving problematic human behaviors (Plumb, Stewart, Dahl and Lundgren, 2009). It is seen that there is very little research about the new approach of ACT approach when the aiming country of our country is screened and it is thought that our country will contribute to the field of psychological counseling with the work done. In the scope of the research, experimental and control groups and preliminary test, post-test and follow-up measurements of 2x3 experimental design were used. The study's study group consists of a total of 24 (12 experimental and 12 control groups) university students studying in different departments and levels, continuing their education in the academic year of 2015-2016 in Ağrı province and İbrahim Chechen University in 2015-2016 academic year. The average age of participants in the experiment and control group is 20. There were 12 participants in the experimental group, 7 female and 5 male, and 12 participants, 7 female and 5 male in the control group. Personal Information Form and Decision Making Style Scale prepared by the researcher were used in the research. In order to decide on the tests to be used in the course of analyzing the data, the scores of the participant's Decision Styles Scale pre-test, which are placed primarily in the experimental and control groups, it was analyzed whether the basic expectations of parametric tests were answered. As a result of the analysis made, the scores, skewness and kurtosis coefficients obtained from the Decision Making Styles Scale were given to the experimental and control groups. It was determined that the distribution was normal in the result of Shapiro-Wilk test, in which the skewness and kurtosis coefficients of each sub-scale were ranked between -1 and +1. Participants in the experimental and control groups; homogeneity test results for decision-style pre-test measurements indicate that the data are homogeneous. According to the results of the Mauchly Globalness Test, it has been determined that working supports the hypothesis. It was determined that there was no significant difference between the pre-test scores obtained from dependent decision-making style of experiment and control groups, but the test group showed lower mean scores at the significant level within the scores of post-test and follow-up tests. Therefore, it can be said that the ACT-oriented psychoeducation program applied to the experimental group reduces the dependent decision-making style scores from the decision style sub-dimensions and the psychoeducation program has a lasting effect. It was determined that there was no significant difference between pre-test, post-test and follow-up scores obtained from the Spontaneous-Instant Decision Style of experiment and control groups. Thus, it can be said that this situation does not cause a significant difference in the Spontaneous-Decision-Making Style scores from the decision style sub-dimensions of the ACT-oriented psychoeducation program applied to the experimental group. The ACT -oriented psychoeducation program had a decline in the intuitive decision-making styles of the individuals, but this decrease did not create significant differences. Thus, it can be said that this situation does not make a meaningful difference in the intuitive decision style scores from the decision style sub-dimensions of the KKT oriented psychoeducation program applied to the experimental group. The pre-test scores obtained from the rational decision-making style of the experimental and control groups showed that there was a difference between the post-test and the follow-up scores, but this difference was not significant. As a result of the analysis, it was determined that the test group had higher levels of rational decision style than the pre - test scores in the post test and follow - up scores, whereas the post test and follow - up test scores in the control group rational decision style showed a decrease compared to the pre - test scores. the pre - test scores. Decision-making Styles Scale Avoidant Decision Making As a result of the analysis of the mean scores of the subscale scores of pre-test, post-test and follow-up measures, the group effect was found to be insignificant. It was determined that the experimental and control groups differed significantly from the pre-test scores obtained from the avoidant decision-making style but did not show any significant change within the scores of the post-test and follow-up tests. [O] . mustafa ercengiz, ali haydar şar 2018

机译：决策是分析有关问题情况的信息并将其与特定结论进行比较的过程，以解决特定的问题（Yıkılmaz，2001; Miller和Byrnes，2001）。决策风格是一种机制，受到内部和外部条件的影响，确定个人决定的方向，决策过程的内容以及决策过程的结果（PAYNE，BETTMAN和BEDNE） Johson，1993;Bavol'ár和奥萨洛瓦，2015）。法案是认知行为治疗家庭的当代成员。行为（验收和承诺治疗）具有与行为疗法和认知疗法的类似和不同方向（Herbet和Forman，2011; Hayes，2004）。 KKT在分析行为中使用存在性和认知方法来响应古典行为治疗方法。 KKT是一个科学翼，试图解决人类问题，旨在解决有问题的人类行为（铅垂，斯图尔特，Dahl和Lundgren，2009）。有人认为，当我们国家的瞄准国家进行筛选时，有关行动方法的新方法几乎没有研究，并且认为我们的国家将为完成工作做出贡献的心理咨询领域。在研究的范围内，使用实验和对照组和初步测试，使用后测试和后续测量的2x3实验设计。该研究的研究小组包括共24名（12个实验和12个对照组）大学学生在不同的部门和水平上学习，在2015-2016学术上继续进行2015-2016的学术年度2015-2016学年年。实验和对照组参与者的平均年龄是20.实验组中有12名参与者，7名女性和5名男性，12名参与者，7名女性和5名男性。研究人员准备的个人信息表格和决策制定风格规模在研究中使用。为了决定在分析数据的过程中使用的测试，参与者的决策风格刻度预测的分数主要在实验和对照组中进行，分析了参数的基本期望吗？测试得到了回答。由于对实验和对照组给出了从决策曲调标度获得的分数，偏移和峰度分子系数。确定该分布是正常的在成熟的-WILK试验结果中，其中每亚级的偏差和刚度系数在-1到+1之间排名。实验和对照组的参与者;决策式预测测量的同质性测试结果表明数据是均匀的。根据毛毛环保试验的结果，已确定工作支持假设。据确定，从依赖决策风格的实验和对照组获得的预测分数之间没有显着差异，但测试组在测试后的分数内显示出较低的平均分数和后续的分数 - 测试。因此，可以说，应用于实验组的面向活动的心理教育程序减少了决策风格子维度的依赖决策风格分数，并且心理教育程序具有持久的效果。据确定，从实验和对照组自发决策风格获得的预测试，测试后和后续评分之间没有显着差异。因此，可以说这种情况不会导致从应用于实验组的活动导向的心理教育程序的决策风格分数的自发决策风格分数显着差异。行为的心理教育计划在个人直观的决策风格下降，但这种减少并没有产生显着的差异。因此，可以说，这种情况不会在从应用于实验组的KKT导向的心理教育程序的决策风格分数中产生有意义的决策风格分数。从实验和对照组的理性决策风格获得的预测分数显示后检验后和随访评分之间存在差异，但这种差异并不重要。由于分析，确定测试组的理性决策风格较高，而不是后测试和后续分数的预测分数，而对照组合理决策风格的后测试和后续测试分数显示出与预测试分数相比的减少。预测分数。决策款式扩展避免决策由于分析了预测试，测试后和后续措施的班次评分的平均分数，发现群体效应是微不足道的。确定实验和对照组从避免决策风格获得的预测评分中显着不同，但在测试后和后续测试的分数内没有显示出任何重大变化。
8. Topological Patterns for Scalable Representation and Analysis of Dataflow Graphs [R] . Sane, N., Kee, H., Seetharaman, G., 2011

机译：可扩展表示的拓扑模式和数据流图的分析

Scalable Analysis of Open Data Graphs

摘要

著录项

相似文献

相关主题

期刊订阅