************************************************************************                                           
                               README
                            ConText v2.00
                            September 2015
  C++ program for convolutional neural networks for text categorization
************************************************************************

0. Change log

* January 2015: beta 0.0 (corresponding to arXiv:1412.1058v1) was released.  
* March 2015: beta 0.1 (corresponding to [1]) was released. 
* July 2015: ConText v1.00 (corresponding to [1]) was released. 
* August 2015: ConText v1.01 (minor changes to v1.00) was released. 
* September 2015: Releasing ConText v2.00 (corresponding to [1] and [2]).  

Contents:
---------
1. Introduction
1.2. Compatibility
1.2. Hardware Requirements
1.3. Software Requirements
2. Download and Installation
3. To Reproduce the Experiments in [1] (supervised)
4. To Reproduce the Experiments in [2] (semi-supervised) 
5. Data Source
6. Documentation
7. Contact
8. Copyright
9. References

---------------
1. Introduction
Software package ConText provides an implementation of convolutional 
neural networks for text categorization described in [1] (supervised CNN) 
and [2] (semi-supervised CNN).  A main difference of v2 from v1 is the 
addition of semi-supervised learning functionality described in [2].  

Unlike the CNN code designed for image data, ConText efficiently handles 
high-dimensional and sparse documents of variable sizes.  It applies CNN 
directly to one-hot vectors representing words (as opposed to requiring 
low-dimensional word vectors learned elsewhere), which leads to internally 
learning an embedding of text regions (or "context" as in the name) directly 
from one-hot vectors.  The benefits of this approach compared with traditional
supervised methods are discussed in [1]. 

The semi-supervised learning functionality is also based on region embedding 
learning.  The merits of this approach compared with previous semi-supervised
methods are discussed in [2]. 

1.1  Compatibility
Files that were produced by ConText v1.xx can be used by ConText v2.00.  
For example, a model file created by ConText v1.xx can be used for making 
predictions by ConText v2.00; input files generated by "prepText gen_regions" 
of v1.xx can be used for training by v2.00.  

However, if you have removed or misplaced a word-mapping file (*.xtext), which
is one of the input files generated by "prepText gen_regions", your training 
would return an error such as: 

  !Input error!: (Detected in AzpData_sparse::_reset_data)
  Unable to open the word-mapping file (data/s.xtext). 

In this case, you would need to re-generate the input files (region/target/
word-mapping files) by calling "prepText gen_regions" again.  
The word-mapping file (*.xtext) defines a mapping between words (or n-grams) 
and dimensions of input region vectors.  While this file was not used for 
training by v1.xx (and therefore, no error even if you had removed it), 
ConText v2.00 uses it at the time of training/testing to ensure the 
consistency of the mapping.

1.2  Hardware Requirement
ConText runs only on graphics processing unit (GPU).  That is, to use ConText, 
your system must have a GPU such as Tesla K20.  Test was done on Tesla K20 and 
Tesla M2070.  

1.3  Software Requirement
The provided makefile uses gcc.  CUDA must be installed.  CUDA v5.0 or higher 
is recommended.  Whether a version earlier than v5.0 works or not is unknown.  

Test was done with: 
  - Ubuntu 12.04.1, CUDA v5.0 
  - Ubuntu 12.04.5, CUDA v6.5 
  
In principle, the code should compile and run also in other systems (e.g., 
Windows), provided that a GPU and an appropriate version of CUDA are 
installed.  But no guarantee. 
  
To optionally use the provided text preprocessing tools, Perl is required.  

----------------------------
2. Download and Installation
Download the package and extract the content.  

A make file "makefile" is provided at the top directory.  You may need to 
customize it for your environment.  In particular, note that the "arch" and 
"code" must be right for your GPU.  

To make sure the build was done correctly, go to sample/ and enter 

  ./sample.sh

In our system, we get "perf:err,0.1725" (which means error rate 17.25%) in the end.  
You might get slightly different error rate depending on the system, due to 
the difference in floating-point computation and so on. 

Note that to run the sample scripts at sample/, you need to set your current
directory to sample/.  

---------------------------------------------------  
3. To Reproduce the Experiments in [1] (supervised) 
See the shell script files train*.sh such as train_imdb_seq.sh at test/.  
Note that to run the scripts at test/, you need to set the current directory 
to test/.  

The input data for IMDB and part of the input data for Elec are included in 
the package. 

For RCV1, we provide shell scripts, but we cannot provide the text data due 
to the copyright issue. 

To obtain the entire Elec used in [1], which includes training data of 
different sizes, download the data archive elec2.tar.gz from 
http://riejohnson.com/cnn_download.html . 

NOTE: The performance results may slightly vary depending on the system, due 
to the difference in floating-point computation and so on. 

--------------------------------------------------------
4. To Reproduce the Experiments in [2] (semi-supervised) 
Reproduction of the semi-supervised experiments in [2] requires the following steps. 

  step-1. Download unlab_data.tar.gz from http://riejohnson.com/cnn_download.html, 
          which contains unlabeled data and resource files.

  step-2. Go to test/ and extract the content of unlab_data.tar.gz so that 
          the directory test/unlab_data will be created. 

  step-3. Use shell scripts, test/semisup-*.sh. The script names correspond to 
          Table 3 of [2]. 
   

For RCV1, we cannot provide the unlabeled text data due to the copyright issue.

To run the scripts at test/, you need to set the current directory to test/.  

NOTE: The performance results may slightly vary depending on the system, due 
to the difference in floating-point computation and so on. 

--------------  
5. Data Source 
- The IMDB dataset [3] is originally from http://ai.stanford.edu/~amaas/data/sentiment/.  
- Sample data used in the scripts at sample/ were extracted from the IMDB dataset.  
- The Elec dataset was derived from part of the Amazon review dataset [4] at 
http://snap.stanford.edu/data/web-Amazon.html. 

----------------
6. Documentation
Documentation on how to use it is available at: 

   http://riejohnson.com/cnn_download.html#doc
   
The sample scripts at sample/ and the scripts for reproducing the 
experiments in [1] and [2] at test/ may be also useful.  

----------
7. Contact 
riejohnson@gmail.com

------------
8. Copyright
This software is distributed under the GNU public license. Please read the 
file COPYING.

-------------
9. References
[1] Rie Johnson and Tong Zhang.  Effective use of word order for text 
categorization with convolutional neural networks.  NAACL HLT 2015.  

[2] Rie Johnson and Tong Zhang.  Semi-supervised convolutional neural networks
for text categorization via region embedding.  To appear in NIPS 2015.

[3] Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, 
and Christopher Potts.  Learning word vectors for sentiment analysis.  ACL, 
2011. 

[4] Julian McAuley and Jure Leskovec.  Hidden factors and hidden topics: 
understanding rating dimensions with review text.  RecSys, 2013.