Hong Xie

The use of sensors for monitoring livestock has opened up new possibilities for the management of livestock in extensive grazing systems. The work presented in this paper aimed to develop a model for predicting the metabolisable energy intake (MEI) of sheep by using temperature, pitch angle, roll angle, distance, speed, and grazing time data obtained directly from wearable sensors on the sheep. A Deep Belief Network (DBN) algorithm was used to predict MEI, which to our knowledge, has not been attempted previously. The results demonstrated that the DBN method could predict the MEI for sheep using sensor data alone. The mean square error (MSE) values of 4.46 and 20.65 have been achieved using the DBN model for training and testing datasets, respectively. We also evaluated the influential sensor data variables, i.e., distance and pitch angle, for predicting the MEI. Our study demonstrates that the application of machine learning techniques directly to on-animal sensor data presents a substantial opportunity to interpret biological interactions in grazing systems directly from sensor data. We expect that further development and refinement of this technology will catalyse a step-change in extensive livestock management, as wearable sensors become widely used by livestock producers.

Feature Selection (FS) methods have been studied extensively in the literature, and there are a crucial component in machine learning techniques. However, unsupervised text feature selection has not been well studied in document clustering problems. Feature selection could be modelled as an optimization problem due to the large number of possible solutions that might be valid. In this paper, a memetic method that combines Differential Evolution (DE) with Simulated Annealing (SA) for unsupervised FS was proposed. Due to the use of only two values indicating the existence or absence of the feature, a binary version of differential evolution is used. A dichotomous DE was used for the purpose of the binary version, and the proposed method is named Dichotomous Differential Evolution Simulated Annealing (DDESA). This method uses dichotomous mutation instead of using the standard mutation DE to be more effective for binary purposes. The Mean Absolute Distance (MAD) filter was used as the feature subset internal evaluation measure in this paper. The proposed method was compared with other state-of-the-art methods including the standard DE combined with SA, which is named DESA in this paper, using five benchmark datasets. The F-micro, F-macro (F-scores) and Average Distance of Document to Cluster (ADDC) measures were utilized as the evaluation measures. The Reduction Rate (RR) was also used as an evaluation measure. Test results showed that the proposed DDESA outperformed the other tested methods in performing the unsupervised text feature selection.

Recurrent Neural Networks (RNN), Long Short-Term Memory Networks (LSTM), and Memory Networks which contain memory are popularly used to learn patterns in sequential data. Sequential data has long sequences that hold relationships. RNN can handle long sequences but suffers from the vanishing and exploding gradient problems. While LSTM and other memory networks address this problem, they are not capable of handling long sequences (50 or more data points long sequence patterns). Language modelling requiring learning from longer sequences are affected by the need for more information in memory. This paper introduces Long Term Memory network (LTM), which can tackle the exploding and vanishing gradient problems and handles long sequences without forgetting. LTM is designed to scale data in the memory and gives a higher weight to the input in the sequence. LTM avoid overfitting by scaling the cell state after achieving the optimal results. The LTM is tested on Penn treebank dataset, and Text8 dataset and LTM achieves test perplexities of 83 and 82 respectively. 650 LTM cells achieved a test perplexity of 67 for Penn treebank, and 600 cells achieved a test perplexity of 77 for Text8. LTM achieves state of the art results by only using ten hidden LTM cells for both datasets.

One of the fundamental tasks in the management of livestock is to understand their behaviour and use this information to increase livestock productivity and welfare. Developing new and improved methods to classify livestock behaviour based on their daily activities can greatly improve livestock management. In this paper, we propose the use of a hierarchical machine learning method to classify livestock behaviours. We first classify the livestock behaviours into two main behavioural categories. Each of the two categories is then broken down at the next level into more specific behavioural categories. We have tested the proposed methodology using two commonly used classifiers, Random Forest, Support Vector Machine and a newer approach involving Deep Belief Networks. Our results show that the proposed hierarchical classification technique works better than the conventional approach. The experimental studies also show that Deep Belief Networks perform better than the Random Forest and Support Vector Machine for most cases.

Word representations are created using analogy context-based statistics and lexical relations on words. Word representations are inputs for the learning models in Natural Language Understanding (NLU) tasks. However, to understand language, knowing only the context is not sufficient. Reading between the lines is a key component of NLU. Embedding deeper word relationships which are not represented in the context enhances the word representation. This paper presents a word embedding which combines an analogy, context-based statistics using Word2Vec, and deeper word relationships using Conceptnet, to create an expanded word representation. In order to fine-tune the word representation, Self-Organizing Map is used to optimize it. The proposed word representation is compared with semantic word representations using Simlex 999. Furthermore, the use of 3D visual representations has shown to be capable of representing the similarity and association between words. The proposed word representation shows a Spearman correlation score of 0.886 and provided the best results when compared to the current state-of-the-art methods, and exceed the human performance of 0.78.

An Adaptive Crossover Memetic Differential Harmony Search (ACMDHS) method was developed for optimizing document clustering in this paper. Due to the complexity of the documents available today, the allocation of the centroid of the document clusters and finding the optimum clusters in the search space are more complex to deal with. One of the possible enhancements on the document clustering is the use of Harmony Search (HS) algorithm to optimize the search. As HS is highly dependent on its control parameters, a differential version of HS was introduced. In the modified version of HS, the Band Width parameter (BW) has been replaced by another pitch adjustment technique due to the sensitivity of the BW parameter. Thus, the Differential Evolution (DE) mutation was used instead. In this paper the DE crossover was also used with the Differential HS for further search space exploitation, the produced global search is named Crossover DHS (CDHS). Moreover, DE crossover (Cr) and mutation (F) probabilities are dynamically tuned through generations. The Memetic optimization was used to enhance the local search capability of CDHS. The proposed ACMDHS was compared to other document clustering techniques using HS, DHS, and K-means methods. It was also compared to its other two variants which are the Memetic DHS (MDHS) and the Crossover Memetic Differential Harmony Search (CMDHS). Moreover, two state-of-the-art clustering methods were also considered in comparisons, the Chaotic Gradient Artificial Bee Colony (CGABC) and the Differential Evolution Memetic Clustering (DEMC). From the experimental results, it was shown that CMDHS variant (the non-adaptive version of ACMDHS) and ACMDHS were highly competitive while both CMDHS and ACMDHS were superior to all other methods.

Text clustering is an effective way that helps crime investigation through grouping of crime-related documents. This paper proposes a Memetic Algorithm Feature Selection (MAFS) approach to enhance the performance of document clustering algorithms used to partition crime reports and criminal news as well as some benchmark text datasets. Two clustering algorithms have been selected to demonstrate the effectiveness of the proposed MAFS method; they are the k-means and Spherical k-means (Spk). The reason behind using these clustering methods is to observe the performance of these algorithms before and after applying a hybrid FS that uses a Memetic scheme. The proposed MAFS method combines a Genetic Algorithm-based wrapper FS with the Relief-F filter. The performance evaluation was based on the clustering outcomes before and after applying the proposed MAFS method. The test results showed that the performance of both k-means and spk improved after the MAFS.

Deep learning techniques have shown to perform well in Question Answering (QA) tasks. We present a framework that combines Memory Network (MN) and Reinforcement Learning (Q-learning) to perform QA, termed Reinforced MN (R-MN). We investigate the proposed framework by the use of Long Short Term Memory Network (LSTM) and Dynamic Memory Network (DMN). We call them Reinforced LSTM (R-LSTM) and Reinforced DMN (R-DMN), respectively. The input text sequence and question are passed to both MN and Q-Learning. The output of the MN is then fed to Q-Learning as a second input for refinement. The R-MN is trained end-to-end. We evaluated R-MNs on the bAbI 1 K QA dataset for all of the 20 tasks. We achieve superior performance when compared to conventional method of RL, LSTM and the state of the art technique, DMN. Using only half of the training data, both R-LSTM and R-DMN achieved all of the bAbI tasks with high accuracies. The experimental results demonstrated that the proposed framework of combining MN and Q-learning enhances the QA tasks while using less training data.

Collaborative filtering (CF) technique in recommender systems (RS) is a well-known and popular technique that exploits relationships between users or items to make product recommendations to an active user. The effectiveness of existing memory based algorithms depend on the similarity measure that is used to identify nearest neighbours. However, similarity measures utilize only the ratings of co-rated items while computing the similarity between a pair of users or items. In most of the e-commerce applications, the rating matrix is too sparse since even active users of an online system tend to rate only a few items of the entire set of items. Therefore, co-rated items among users are even sparser. Moreover, the ratings a user gives an individual item tells us nothing about his comprehensive interest without which the generated recommendations may not be satisfactory to a user. In order to be able to address these issues, a comprehensive study is made of the various existing measures of similarity in a collaborative filtering recommender system (CFRS) and a hierarchical categorization of products has been proposed to understand the interest of a user in a wider scope so as to provide better recommendations as well as to alleviate data sparsity.

With the wide increase of the volume of electronic documents, it becomes inevitable the need to invent more sophisticated machine learning methods to manage the issue. In this paper, a Memetic feature selection technique is proposed to improve the k-means and the spherical k-means clustering algorithms. The proposed Memetic feature selection technique combines the wrapper inductive method with the filter ranking method. The internal and external clustering evaluation measures are used to assess the resulted clusters. The test results showed that after using the proposed hybrid method, the resulted clusters were more accurate and more compacted in comparison to the clusters resulted from using the GA-selected feature or using the entire feature space.