Hong Xie

Recurrent Neural Networks (RNN), Long Short-Term Memory Networks (LSTM), and Memory Networks which contain memory are popularly used to learn patterns in sequential data. Sequential data has long sequences that hold relationships. RNN can handle long sequences but suffers from the vanishing and exploding gradient problems. While LSTM and other memory networks address this problem, they are not capable of handling long sequences (50 or more data points long sequence patterns). Language modelling requiring learning from longer sequences are affected by the need for more information in memory. This paper introduces Long Term Memory network (LTM), which can tackle the exploding and vanishing gradient problems and handles long sequences without forgetting. LTM is designed to scale data in the memory and gives a higher weight to the input in the sequence. LTM avoid overfitting by scaling the cell state after achieving the optimal results. The LTM is tested on Penn treebank dataset, and Text8 dataset and LTM achieves test perplexities of 83 and 82 respectively. 650 LTM cells achieved a test perplexity of 67 for Penn treebank, and 600 cells achieved a test perplexity of 77 for Text8. LTM achieves state of the art results by only using ten hidden LTM cells for both datasets.

Word representations are created using analogy context-based statistics and lexical relations on words. Word representations are inputs for the learning models in Natural Language Understanding (NLU) tasks. However, to understand language, knowing only the context is not sufficient. Reading between the lines is a key component of NLU. Embedding deeper word relationships which are not represented in the context enhances the word representation. This paper presents a word embedding which combines an analogy, context-based statistics using Word2Vec, and deeper word relationships using Conceptnet, to create an expanded word representation. In order to fine-tune the word representation, Self-Organizing Map is used to optimize it. The proposed word representation is compared with semantic word representations using Simlex 999. Furthermore, the use of 3D visual representations has shown to be capable of representing the similarity and association between words. The proposed word representation shows a Spearman correlation score of 0.886 and provided the best results when compared to the current state-of-the-art methods, and exceed the human performance of 0.78.

Text clustering is an effective way that helps crime investigation through grouping of crime-related documents. This paper proposes a Memetic Algorithm Feature Selection (MAFS) approach to enhance the performance of document clustering algorithms used to partition crime reports and criminal news as well as some benchmark text datasets. Two clustering algorithms have been selected to demonstrate the effectiveness of the proposed MAFS method; they are the k-means and Spherical k-means (Spk). The reason behind using these clustering methods is to observe the performance of these algorithms before and after applying a hybrid FS that uses a Memetic scheme. The proposed MAFS method combines a Genetic Algorithm-based wrapper FS with the Relief-F filter. The performance evaluation was based on the clustering outcomes before and after applying the proposed MAFS method. The test results showed that the performance of both k-means and spk improved after the MAFS.

Collaborative filtering (CF) technique in recommender systems (RS) is a well-known and popular technique that exploits relationships between users or items to make product recommendations to an active user. The effectiveness of existing memory based algorithms depend on the similarity measure that is used to identify nearest neighbours. However, similarity measures utilize only the ratings of co-rated items while computing the similarity between a pair of users or items. In most of the e-commerce applications, the rating matrix is too sparse since even active users of an online system tend to rate only a few items of the entire set of items. Therefore, co-rated items among users are even sparser. Moreover, the ratings a user gives an individual item tells us nothing about his comprehensive interest without which the generated recommendations may not be satisfactory to a user. In order to be able to address these issues, a comprehensive study is made of the various existing measures of similarity in a collaborative filtering recommender system (CFRS) and a hierarchical categorization of products has been proposed to understand the interest of a user in a wider scope so as to provide better recommendations as well as to alleviate data sparsity.

With the wide increase of the volume of electronic documents, it becomes inevitable the need to invent more sophisticated machine learning methods to manage the issue. In this paper, a Memetic feature selection technique is proposed to improve the k-means and the spherical k-means clustering algorithms. The proposed Memetic feature selection technique combines the wrapper inductive method with the filter ranking method. The internal and external clustering evaluation measures are used to assess the resulted clusters. The test results showed that after using the proposed hybrid method, the resulted clusters were more accurate and more compacted in comparison to the clusters resulted from using the GA-selected feature or using the entire feature space.

It is challenging to ensure security and to minimize economic impacts due to cyber-attacks because of the heavy reliance on ICT in different organizations and this paper presents an approach to estimate the cost of cyber security in public and private sector organizations. The paper also describes an approach for selecting the type of cyber security improvements to ensure that organizational goals are achieved. Different types of cyber-attacks and the subsequent impacts of these attacks are considered. A Value Analysis method is proposed to support the decision-making process by determining the priorities of deployment of various cyber security technologies. The proposed method is based on security costs related to and the losses due to attacks. Examples are provided in the paper to illustrate the proposed approach.

In universities, the amount of Internet traffic fluctuates over the time of a day and over the period of a semester. With the limited bandwidth resource available and ever increasing demand for high throughput due to multi-media, the Internet bandwidth have to be managed within an organization so that the priority traffic critical to the business is not slowed down by less critical, often entertainment, traffic. An important aspect in setting the policy of the bandwidth management is the knowledge of the Internet usage patterns over a period of time. The ability to predict the usage patterns provides the policy maker a powerful tool to set the rules and policies such that the allocation of Internet bandwidth is conducted dynamically in favor of business critical traffic while at the same time aiming to serve all users subject to the availability of the total available bandwidth. This paper reports the initial experiment of predicting the Internet usage patterns using RapidMiner Machine Learning tools with SVM algorithm for the IT Service Department of Sanata Dharma University in Indonesia. The algorithm used the dataset captured from the university over a period of time as the basis for prediction. The result of the dataset analysis is intended to the basis of policies development relating to bandwidth management.

The main goal of e-government implementation is to improve the effectiveness, efficiency and quality of public service delivery using Information and Communication Technologies (ICT). However, its success is dependent on the provision of information security goals such as confidentiality, integrity, availability and trust. Therefore, cybersecurity is vital for the successful adoption of e-government systems. This paper presents an assessment of cybersecurity practices, cyber threats and other factors affecting effective implementation of cybersecurity program in government organizations in Bhutan. Selected cybersecurity practices included in the study were cyber policy, risk management, training and awareness, and access controls for protection of network including mobile computing devices. Out of 280 potential respondents, 157 respondents completed the survey. The results show that, in many organizations, there is very limited use of or a lack of formal cybersecurity policy, risk management, awareness, or incident management practices. The results also indicate that many organizations have either suffered from, or been affected by, cybersecurity threats such as malware, hacking and phishing scams. The study recommends both managerial and technological practices to improve cybersecurity posture of government organizations and to improve people’s level of trust and confidence in e-government services.

This article presents findings from a Content Analysis of verbal and audio protocol of Mental Model study. When using touch screen display with audio feedback, it was found that the users' mental model is in one dimensional as using screen-reader program. Previous experience using screen-reader program may affected the users' experience using a touch screen with audio feedback. In addition, investigation of blind users' spatial ability on users' performance was also conducted. This study revealed that there was a significant interaction effect between the user's spatial ability and the time taken to answer a question using touch display with audio feedback.

The purpose of this study is to discover in detail the two-dimensional mental model created by the blind people using touch screen with audio feedback. It is hypothesized that by augmenting a touch screen display with audio feedback, blind user would be able to have two-dimensional perspective in their mental model. If their mental model of a web page is two-dimensional, the blind people should be able to gain an overview of the web pages accurately, which is important for navigation of the web page. This paper discusses an on-going study to elicit mental models from the blind users using diagrammatic epresentation. The study so far revealed that there is a significant difference between performances using screen-reader program and touch-screen display with audio feedback. However, the performance seems to be affected by page complexity.