[TeamTalk 308]: [844] trunk/TeamTalk/Resources/Grammar: Tokenize with underscores since that's what phoenix understands.

[tk@edam.speech.cs.cmu.edu]

An HTML attachment was scrubbed...
URL: http://mailman.srv.cs.cmu.edu/pipermail/teamtalk-developers/attachments/20071017/d6856c8b/attachment.html
-------------- next part --------------
Modified: trunk/TeamTalk/Resources/Grammar/GRAMMAR/TeamTalkTask.gra
===================================================================
--- trunk/TeamTalk/Resources/Grammar/GRAMMAR/TeamTalkTask.gra	2007-10-17 18:56:22 UTC (rev 843)
+++ trunk/TeamTalk/Resources/Grammar/GRAMMAR/TeamTalkTask.gra	2007-10-18 02:35:42 UTC (rev 844)
@@ -267,10 +267,10 @@
 ;
 
 [Side]
-	(the=north=end=of)
-	(the=east=end=of)
-	(the=south=end=of)
-	(the=west=end=of)
+	(the_north_end_of)
+	(the_east_end_of)
+	(the_south_end_of)
+	(the_west_end_of)
 
 [Goal]
 	([Home])
@@ -371,18 +371,10 @@
 	(do not)
 MOD
 	(thanks)
-	(thank=you)     # should be a lexeme
+	(thank_you)     # should be a lexeme
 	(not really)
 	(it's not)
 	(i'm not)
-#NO
-#	(no)
-#	(not)
-#GOOD
-#	(right)
-#	(correct)
-#	(good)
-#	(okay)
 ;
 
 

Modified: trunk/TeamTalk/Resources/Grammar/tokenize.pl
===================================================================
--- trunk/TeamTalk/Resources/Grammar/tokenize.pl	2007-10-17 18:56:22 UTC (rev 843)
+++ trunk/TeamTalk/Resources/Grammar/tokenize.pl	2007-10-18 02:35:42 UTC (rev 844)
@@ -81,7 +81,7 @@
       $text = $line; $prob = undef;
     }
     $text =~ s/^\s*\((.+?)\)\s*$/$1/;  # trim spaces from ends, strip ()'s
-    $text =~ s/\s+/=/g;  # tokenize the text by substituting spaces
+    $text =~ s/\s+/_/g;  # tokenize the text by substituting spaces
     $tokens{"$text:$classid"}++;
     $lexset{"$text:$classid"} = $prob;
   }