Search results

This paper aims to provide knowledge about methods used to evaluate the accessibility of Iranian web sites through a real research environment.

The main theoretical base of this article is that almost all research conducted to evaluate the accessibility of web sites has used only one of three methods (namely manual, automatic and users' experiences). However, it seems that using a combination of these three methods is actually necessary. Consequently, taking a critical approach this article examines 18 Iranian ministerial web sites using the three methods.

Findings in both documents and evaluative studies show that each of the methods has deficits and it is necessary to employ a combination of the methods in order to conduct a reliable accessibility assessment. The different rankings of ministerial web sites in various evaluations have confirmed this view.

Identifying the advantages and disadvantages of the three methods and employing them in real research environments to assess the accessibility of ministerial web sites has been a useful exercise. The work can not only be a starting point for further discussions on the reliability of accessibility evaluation methods, but can also be applied to other types of web site.

This study aims to focus on the evaluation of systems for the automatic translation of questions destined to translingual question‐answer (QA) systems. The efficacy of online translators when performing as tools in QA systems is analysed using a collection of documents in the Spanish language.

Automatic translation is evaluated in terms of the functionality of actual translations produced by three online translators (Google Translator, Promt Translator, and Worldlingo) by means of objective and subjective evaluation measures, and the typology of errors produced was identified. For this purpose, a comparative study of the quality of the translation of factual questions of the CLEF collection of queries was carried out, from German and French to Spanish.

It was observed that the rates of error for the three systems evaluated here are greater in the translations pertaining to the language pair German‐Spanish. Promt was identified as the most reliable translator of the three (on average) for the two linguistic combinations evaluated. However, for the Spanish‐German pair, a good assessment of the Google online translator was obtained as well. Most errors (46.38 percent) tended to be of a lexical nature, followed by those due to a poor translation of the interrogative particle of the query (31.16 percent).

The evaluation methodology applied focuses above all on the finality of the translation. That is, does the resulting question serve as effective input into a translingual QA system? Thus, instead of searching for “perfection”, the functionality of the question and its capacity to lead one to an adequate response are appraised. The results obtained contribute to the development of improved translingual QA systems.

Reliability and political bias of mass media has been a controversial topic in the literature. The purpose of this paper is to propose and implement a methodology for fully automatic evaluation of the political tendency of the written media on the web, which does not rely on subjective human judgments.

The underlying idea is to base the evaluation on fully automatic comparison of the texts of articles on different news websites to the overtly political texts with known political orientation. The authors also apply an alternative approach for evaluation of political tendency based on wisdom of the crowds.

The authors found that the learnt classifier can accurately distinguish between self-declared left and right news sites. Furthermore, news sites’ political tendencies can be identified by automatic classifier learnt from manifestly political texts without recourse to any manually tagged data. The authors also show a high correlation between readers’ perception (as a “wisdom of crowds” evaluation) of the bias and the classifier results for different news sites.

The results are quite promising and can put an end to the never ending dispute on the reliability and bias of the press.

This paper proposes and implements a new approach for fully automatic (independent of human opinion/assessment) identification of political bias of news sites by their texts.

Defects in concrete surfaces are inevitably recurring during construction, which needs to be checked and accepted during construction and completion. Traditional manual inspection of surface defects requires inspectors to judge, evaluate and make decisions, which requires sufficient experience and is time-consuming and labor-intensive, and the expertise cannot be effectively preserved and transferred. In addition, the evaluation standards of different inspectors are not identical, which may lead to cause discrepancies in inspection results. Although computer vision can achieve defect recognition, there is a gap between the low-level semantics acquired by computer vision and the high-level semantics that humans understand from images. Therefore, computer vision and ontology are combined to achieve intelligent evaluation and decision-making and to bridge the above gap.

Combining ontology and computer vision, this paper establishes an evaluation and decision-making framework for concrete surface quality. By establishing concrete surface quality ontology model and defect identification quantification model, ontology reasoning technology is used to realize concrete surface quality evaluation and decision-making.

Computer vision can identify and quantify defects, obtain low-level image semantics, and ontology can structurally express expert knowledge in the field of defects. This proposed framework can automatically identify and quantify defects, and infer the causes, responsibility, severity and repair methods of defects. Through case analysis of various scenarios, the proposed evaluation and decision-making framework is feasible.

This paper establishes an evaluation and decision-making framework for concrete surface quality, so as to improve the standardization and intelligence of surface defect inspection and potentially provide reusable knowledge for inspecting concrete surface quality. The research results in this paper can be used to detect the concrete surface quality, reduce the subjectivity of evaluation and improve the inspection efficiency. In addition, the proposed framework enriches the application scenarios of ontology and computer vision, and to a certain extent bridges the gap between the image features extracted by computer vision and the information that people obtain from images.

The aim of this study is to explore students' expectations and perceived effectiveness of computer-assisted review tools, and the differences in reliability and validity between human evaluation and automatic evaluation, to find a way to improve students' English writing ability.

Based on the expectancy disconfirmation theory (EDT) and Intelligent Computer-Assisted Language Learning (ICALL) theory, an experiment is conducted through the observation method, semistructured interview method and questionnaire survey method. In the experiment, respondents were asked to write and submit four essays on three online automated essay evaluation (AEE) systems in total, one essay every two weeks. Also, two teacher raters were invited to score the first and last papers of each student. The respondents' feedbacks were investigated to confirm the effectiveness of the AEE system; the evaluation results of the AEE systems and teachers were compared; descriptive statistics was used to analyze the experimental data.

The experiment revealed that the respondents held high expectations for the computer-assisted evaluation tools, and the effectiveness of computer scoring feedback on students was higher than that of teacher scoring feedback. Moreover, at the end of the writing project, the students' independent learning ability and English writing ability were significantly improved. Besides, there was a positive correlation between students' initial expectations of computer-assisted learning tools and the final evaluation of learning results.

The innovation lies in the use of observation methods, questionnaire survey methods, data analysis, and other methods for the experiment, and the combination of deep learning theory, EDT and descriptive statistics, which has particular reference value for future works.

Document retrival systems are designed and implemented for the purpose of satisfying the information needs of a group of users, and the final stage of the technical processes required is the search of the stored file of documents or document surrogates. Studies of many aspects of retrieval systems have been made, but work has concentrated on the indexing or input stage, and little attention has been paid to the search or output stage.

The purpose of this paper is to describe the aspects to be considered when evaluating web search engines' accessibility for people with disabilities. The authors provide an overview of related work and outline a theoretical framework for a comprehensive accessibility study of web search engines, regarding the principles of disability studies and the idea of inclusion.

The paper is based on a literature review, and an aggregation of recommended actions in practice, mainly the W3C Web Accessibility Initiative's (WAI) evaluation model.

A good way to conduct an accessibility study in a comprehensive manner is the WAI methodology consisting of three‐steps: preliminary review to quickly identify potential accessibility problems; conformance evaluation to determine whether a website meets established accessibility standards; and user testing to include real people with disabilities in a practical use. For the use case “web search engines” some special issues have to be taken into consideration.

The paper can be seen as a brainstorming and describes a theoretical concept of how to do. Conclusions about actual barriers of web search engines and criteria of satisfaction for people with disabilities do not exist as of yet; the model is not tested so far.

This paper provides practical implications for researchers who want to conduct an accessibility study, especially of web search engines. Findings of such studies can have practical implications for web search engine developers to improve accessibility of their product. The accessibility of web search engines does not only have implications for people with special needs, but also for the elderly or temporarily handicapped people.

This paper combines findings from web search engine research with aspects of disability studies. Therefore, it provides insights for researches, search engine developers and educators in practice on how important accessibility of web search engines for people with disabilities is, how it can be measured and what aspects need to be considered.

Semantic retrieval and discovery of datasets published as open data remains a challenging task. The datasets inherently originate in the globally distributed web jungle, lacking the luxury of centralized database administration, database schemes, shared attributes, vocabulary, structure and semantics. The existing dataset catalogs provide basic search functionality relying on keyword search in brief, incomplete or misleading textual metadata attached to the datasets. The search results are thus often insufficient. However, there exist many ways of improving the dataset discovery by employing content-based retrieval, machine learning tools, third-party (external) knowledge bases, countless feature extraction methods and description models and so forth.

In this paper, the authors propose a modular framework for rapid experimentation with methods for similarity-based dataset discovery. The framework consists of an extensible catalog of components prepared to form custom pipelines for dataset representation and discovery.

The study proposes several proof-of-concept pipelines including experimental evaluation, which showcase the usage of the framework.

To the best of authors’ knowledge, there is no similar formal framework for experimentation with various similarity methods in the context of dataset discovery. The framework has the ambition to establish a platform for reproducible and comparable research in the area of dataset discovery. The prototype implementation of the framework is available on GitHub.

With the growth in popularity of academic social networking sites, evaluating the quality of the academic information they contain has become increasingly important. Users' evaluations of this are based on predefined criteria, with external factors affecting how important these are seen to be. As few studies on these influences exist, this research explores the factors affecting the importance of criteria used for judging high-quality answers on academic social Q&A sites.

Scholars who had recommended answers on ResearchGate Q&A were asked to complete a questionnaire survey to rate the importance of various criteria for evaluating the quality of these answers. Statistical analysis methods were used to analyze the data from 215 questionnaires to establish the influence of scholars' demographic characteristics, the question types, the discipline and the combination of these factors on the importance of each evaluation criterion.

Particular disciplines and academic positions had a significant impact on the importance ratings of the criteria of relevance, completeness and credibility. Also, some combinations of factors had a significant impact: for example, older scholars tended to view verifiability as more important to the quality of answers to information-seeking questions than to discussion-seeking questions within the LIS and Art disciplines.

This research can help academic social Q&A platforms recommend high-quality answers based on different influencing factors, in order to meet the needs of scholars more effectively.

Purpose — Ranking is a natural task for a search engine; a search engine result page is the most common example. This chapter aims at illustrating the motivations and the concepts of rank correlation in a practical way for the researchers active in the different domains of search engines.

Methodology/approach — To this end, this chapter provides a survey according to a topic-oriented basis of the search engine evaluation literature specifically devoted to or based on rank correlation; the chapter explains and illustrates how statistics is the only approach to rank correlation.

Findings/research limitations/implications — The chapter introduces the pros and cons of rank correlation measures through a light-weight formal description and a number of concrete examples to find the measure that better fit a context.

Practical implications — This chapter provides a blueprint for the application of rank correlation within scientific experimentation or item/service recommendation.

Social implications — Rank correlation analyses impact on the success or failure of a search engine in performing the tasks for which it has been designed and hence on the people's daily life activities.

Originality/value of paper — This chapter places rank correlation within a scientific research perspective and in particular connects to and complements documentation on search engine evaluation.