Skip to content

Commit 30b1478

Browse files
kk7nckk7nc
authored
Update README.rst
1 parent 7fd5a70 commit 30b1478

1 file changed

Lines changed: 130 additions & 0 deletions

File tree

‎README.rst‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1044,6 +1044,136 @@ output:
10441044
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
10451045
Conditional Random Field (CRF)
10461046
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1047+
Conditional Random Field (CRF) is an undirected graphical model as shown in figure. CRFs state the conditional probability of a label sequence *Y* give a sequence of observation *X* *i.e.* P(Y|X). CRFs can incorporate complex features of observation sequence without violating the independence assumption by modeling the conditional probability of the label sequence rather than the joint probability P(X,Y). The concept of clique which is a fully connected subgraph and clique potential are used for computing P(X|Y). Considering one potential function for each clique of the graph, the probability of a variable configuration is corresponding to the product of a series of non-negative potential function. The value computed by each potential function is equivalent to the probability of the variables in its corresponding clique taken on a particular configuration.
1048+
1049+
1050+
.. image:: docs/pic/CRF.png
1051+
1052+
1053+
Example from `Here <http://sklearn-crfsuite.readthedocs.io/en/latest/tutorial.html>`__
1054+
Let’s use CoNLL 2002 data to build a NER system
1055+
CoNLL2002 corpus is available in NLTK. We use Spanish data.
1056+
1057+
1058+
.. code:: python
1059+
1060+
import nltk
1061+
import sklearn_crfsuite
1062+
from sklearn_crfsuite import metrics
1063+
nltk.corpus.conll2002.fileids()
1064+
train_sents = list(nltk.corpus.conll2002.iob_sents('esp.train'))
1065+
test_sents = list(nltk.corpus.conll2002.iob_sents('esp.testb'))
1066+
1067+
1068+
sklearn-crfsuite (and python-crfsuite) supports several feature formats; here we use feature dicts.
1069+
1070+
.. code:: python
1071+
1072+
def word2features(sent, i):
1073+
word = sent[i][0]
1074+
postag = sent[i][1]
1075+
1076+
features = {
1077+
'bias': 1.0,
1078+
'word.lower()': word.lower(),
1079+
'word[-3:]': word[-3:],
1080+
'word[-2:]': word[-2:],
1081+
'word.isupper()': word.isupper(),
1082+
'word.istitle()': word.istitle(),
1083+
'word.isdigit()': word.isdigit(),
1084+
'postag': postag,
1085+
'postag[:2]': postag[:2],
1086+
}
1087+
if i > 0:
1088+
word1 = sent[i-1][0]
1089+
postag1 = sent[i-1][1]
1090+
features.update({
1091+
'-1:word.lower()': word1.lower(),
1092+
'-1:word.istitle()': word1.istitle(),
1093+
'-1:word.isupper()': word1.isupper(),
1094+
'-1:postag': postag1,
1095+
'-1:postag[:2]': postag1[:2],
1096+
})
1097+
else:
1098+
features['BOS'] = True
1099+
1100+
if i < len(sent)-1:
1101+
word1 = sent[i+1][0]
1102+
postag1 = sent[i+1][1]
1103+
features.update({
1104+
'+1:word.lower()': word1.lower(),
1105+
'+1:word.istitle()': word1.istitle(),
1106+
'+1:word.isupper()': word1.isupper(),
1107+
'+1:postag': postag1,
1108+
'+1:postag[:2]': postag1[:2],
1109+
})
1110+
else:
1111+
features['EOS'] = True
1112+
1113+
return features
1114+
1115+
1116+
def sent2features(sent):
1117+
return [word2features(sent, i) for i in range(len(sent))]
1118+
1119+
def sent2labels(sent):
1120+
return [label for token, postag, label in sent]
1121+
1122+
def sent2tokens(sent):
1123+
return [token for token, postag, label in sent]
1124+
1125+
X_train = [sent2features(s) for s in train_sents]
1126+
y_train = [sent2labels(s) for s in train_sents]
1127+
1128+
X_test = [sent2features(s) for s in test_sents]
1129+
y_test = [sent2labels(s) for s in test_sents]
1130+
1131+
1132+
To see all possible CRF parameters check its docstring. Here we are useing L-BFGS training algorithm (it is default) with Elastic Net (L1 + L2) regularization.
1133+
1134+
1135+
1136+
.. code:: python
1137+
1138+
crf = sklearn_crfsuite.CRF(
1139+
algorithm='lbfgs',
1140+
c1=0.1,
1141+
c2=0.1,
1142+
max_iterations=100,
1143+
all_possible_transitions=True
1144+
)
1145+
crf.fit(X_train, y_train)
1146+
1147+
1148+
Evaluation
1149+
1150+
1151+
.. code:: python
1152+
1153+
y_pred = crf.predict(X_test)
1154+
print(metrics.flat_classification_report(
1155+
y_test, y_pred, digits=3
1156+
))
1157+
1158+
1159+
Output:
1160+
1161+
.. code:: python
1162+
1163+
precision recall f1-score support
1164+
1165+
B-LOC 0.810 0.784 0.797 1084
1166+
B-MISC 0.731 0.569 0.640 339
1167+
B-ORG 0.807 0.832 0.820 1400
1168+
B-PER 0.850 0.884 0.867 735
1169+
I-LOC 0.690 0.637 0.662 325
1170+
I-MISC 0.699 0.589 0.639 557
1171+
I-ORG 0.852 0.786 0.818 1104
1172+
I-PER 0.893 0.943 0.917 634
1173+
O 0.992 0.997 0.994 45355
1174+
1175+
avg / total 0.970 0.971 0.971 51533
1176+
10471177
10481178
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
10491179
Deep Learning

0 commit comments

Comments
 (0)