at_grammar_induction.git
2 years agoFixed broken escapes in README.md master
YeGoblynQueenne@splinter [Fri, 25 Nov 2016 13:12:26 +0000]
Fixed broken escapes in README.md
* 'Twas my fault after all.

2 years agoRewrote README.md; still needs work.
YeGoblynQueenne@splinter [Fri, 25 Nov 2016 12:30:29 +0000]
Rewrote README.md; still needs work.
* I seem to be getting the escaping of underscores wrong? Or is that a
  problem with the way Swi (or Firefox) renders the escapes? Needs
  pickign at.

2 years agoGave some tlc to structured comments in module configuration.
YeGoblynQueenne@splinter [Fri, 25 Nov 2016 12:28:11 +0000]
Gave some tlc to structured comments in module configuration.

2 years agoMoved doc_browser/0 before module loading.
YeGoblynQueenne@splinter [Fri, 25 Nov 2016 11:57:11 +0000]
Moved doc_browser/0 before module loading.
* If you try to load the modules first the doc browser doesn't find
  anything to render, apparently.

2 years agoReduced whitespace.
YeGoblynQueenne@splinter [Fri, 25 Nov 2016 11:56:40 +0000]
Reduced whitespace.

2 years agoGave some tlc to structured comments of predicates, module declarations.
YeGoblynQueenne@splinter [Tue, 30 Aug 2016 11:35:09 +0000]
Gave some tlc to structured comments of predicates, module declarations.
* Still can't get structured comments to always display correctly in
  Swi documentation browser.

2 years agoAdded a few comments to corpus_tools (mtgjson processing) predicates.
YeGoblynQueenne@splinter [Tue, 30 Aug 2016 07:51:53 +0000]
Added a few comments to corpus_tools (mtgjson processing) predicates.

2 years agoUpdated python query interface file to new project structure.
YeGoblynQueenne@splinter [Tue, 30 Aug 2016 06:51:54 +0000]
Updated python query interface file to new project structure.
* Not that this is used anywhere right now but who knows.

2 years agoAdded reload/0, to make project, reload examples set easily.
YeGoblynQueenne@splinter [Tue, 30 Aug 2016 06:48:12 +0000]
Added reload/0, to make project, reload examples set easily.

2 years agoRenamed tree_learning dir --> grammar_learning (because duh)
YeGoblynQueenne@splinter [Tue, 30 Aug 2016 06:38:32 +0000]
Renamed tree_learning dir --> grammar_learning (because duh)

2 years agoMoved project load file to root dir.
YeGoblynQueenne@splinter [Tue, 30 Aug 2016 06:08:14 +0000]
Moved project load file to root dir.

2 years agoRemoved old grammar evaluation predicates, config option.
YeGoblynQueenne@splinter [Tue, 30 Aug 2016 05:51:14 +0000]
Removed old grammar evaluation predicates, config option.

2 years agoAdded 'lex' suffix to child bootstrapped grammars' names.
YeGoblynQueenne@splinter [Tue, 30 Aug 2016 05:43:15 +0000]
Added 'lex' suffix to child bootstrapped grammars' names.

2 years agoAdded dynamic config output to bootstrap parent (already in children)
YeGoblynQueenne@splinter [Mon, 29 Aug 2016 18:14:11 +0000]
Added dynamic config output to bootstrap parent (already in children)

2 years agoRemoved redundant lexicalisation_strategy/1 options (in comments only)
YeGoblynQueenne@splinter [Mon, 29 Aug 2016 18:05:52 +0000]
Removed redundant lexicalisation_strategy/1 options (in comments only)

2 years agoCorrected conditional printing of dynamic lexicalisation config option.
YeGoblynQueenne@splinter [Mon, 29 Aug 2016 18:01:54 +0000]
Corrected conditional printing of dynamic lexicalisation config option.

2 years agoRemoved some configuration options form dynamic list.
YeGoblynQueenne@splinter [Mon, 29 Aug 2016 08:36:43 +0000]
Removed some configuration options form dynamic list.
* evaluation_parsing_time_limit/1 is only relevant to testing, so let's
  leave it aside for the moment until I have some sort of environment
  variable that tells us when specific options are relevant (and
  therefore should be reported to the user) or not.
* output_type/1 I have never edited dynamically, I believe. I guess it
  would be nice to have and all but one point of the dynamic
  configuration predicates is to make it easier to figure out what is
  the minimum set of configuration options needed to run the program, so
  it's a good idea to unclutter often.

2 years agoAdded dynamic configuration printing to output grammars.
YeGoblynQueenne@splinter [Mon, 29 Aug 2016 08:27:27 +0000]
Added dynamic configuration printing to output grammars.
* Dynamic config options increasingly look like just a short version of
  the full config options- since I've stopped using them actually
  dynamically a while ago now.

2 years agoFiddling about with corpus size in grammar evaluation.
YeGoblynQueenne@splinter [Fri, 26 Aug 2016 11:58:15 +0000]
Fiddling about with corpus size in grammar evaluation.
* Was trying to make the training corpus small enough to allow nested
  k-fold cross validation with entire M:tG corpus, so there's a few
  hacks left in commented out. I think I mostly put everything back the
  way it was before, besides that and the refactoring to recall/4 (see
  next).
* Changed recall/4 to use findall/3 loops rather than aggregate/4,
  because I'm having some issues with nondeterminism with that.
* Renamed stochastic_gnf_parsing_time_limit/1 to
  evaluation_parsing_time_limit/1 (since it's now used for rr grammars
  also, and deterministic ones too, both rr and gnf).

2 years agoAdded some corpus statistics predicates.
YeGoblynQueenne@splinter [Thu, 18 Aug 2016 18:28:27 +0000]
Added some corpus statistics predicates.
* Some of the new predicates in corpus_utilities actually checks the
  currently configured corpus instead of taking a corpus (ie, a set of
  examples) as an argument. The latter is probably a pattern I want to
  use. Probably. Passing the current examples corpus and then
  pretty-printing should be left to project_utilities predicates (these
  already handle the pretty-printing for the latest stats preds).

2 years agoReverted change to dcg_derivation/3, that made it deterministic ish.
YeGoblynQueenne@splinter [Sat, 13 Aug 2016 08:35:50 +0000]
Reverted change to dcg_derivation/3, that made it deterministic ish.
* I didn't comment that change but I think it was causing
  dcg_derivation/3 to go nondeterministic at times it really shouldn't.
  On the other hand, now we're trying to find parses and only find one,
  so that can't work either. Reverting for now and we'll see how it
  goes.

2 years agoPred to capitalise the first character of atoms.
YeGoblynQueenne@splinter [Sat, 13 Aug 2016 08:33:42 +0000]
Pred to capitalise the first character of atoms.

2 years agoUpdated gnf parsing limit option name in dynamic config printing.
YeGoblynQueenne@splinter [Sat, 13 Aug 2016 08:33:10 +0000]
Updated gnf parsing limit option name in dynamic config printing.

2 years agoCouple of vim scripts used to clean up logging etc.
YeGoblynQueenne@splinter [Sat, 13 Aug 2016 08:31:03 +0000]
Couple of vim scripts used to clean up logging etc.
* These are just useful for my dissertation- probably remove later.

2 years agoRe-did precision & recall and added to k-fold cross-validation.
YeGoblynQueenne@splinter [Thu, 11 Aug 2016 20:07:26 +0000]
Re-did precision & recall and added to k-fold cross-validation.
* Also changed default ish deterministic metric to f_score.

2 years agoAdded predicates to count/comput min/max/average corpus sentence length.
YeGoblynQueenne@splinter [Thu, 11 Aug 2016 15:50:48 +0000]
Added predicates to count/comput min/max/average corpus sentence length.

2 years agoMade dynamic configuration listing of bootstrapping options conditional.
YeGoblynQueenne@splinter [Thu, 11 Aug 2016 12:07:45 +0000]
Made dynamic configuration listing of bootstrapping options conditional.
* Conditional upon the number of bags being not 0, that is. If it's
  negative, hilarity will ensue of course, but why prevent this?

2 years agoRefactored sentence frequencies printing.
YeGoblynQueenne@splinter [Wed, 10 Aug 2016 10:05:28 +0000]
Refactored sentence frequencies printing.
* This is not very useful right now but I was trying to figure out if I
  can avoid having to count sentence frequencies "on the fly" during
  cross-validation, and instead load them from a file. I can't because
  I need to calculate relative frequency of each sentence on a current
  fold which means I'd have to re-calculate them for each fold every
  time.
* Well I mean, I _can_ do that but it sounds like too much hassle and
  I'm not sure it will really save all that much time. I'll need some
  benchmarks so not now.

2 years agoMade nested_k_fold_cross_validation/0 raise error when k <= 1.
YeGoblynQueenne@splinter [Wed, 10 Aug 2016 09:46:46 +0000]
Made nested_k_fold_cross_validation/0 raise error when k <= 1.
* Because how exactly are you cross-validating if you're training and
  testing on the same set?

2 years agoFixed nested k-fold cross-validation; added pretty highlights vim script.
YeGoblynQueenne@splinter [Tue, 9 Aug 2016 16:46:37 +0000]
Fixed nested k-fold cross-validation; added pretty highlights vim script.

2 years agoMade k-fold cross-validation tuning reset parameters at the end. WIP y.
YeGoblynQueenne@splinter [Tue, 9 Aug 2016 09:41:12 +0000]
Made k-fold cross-validation tuning reset parameters at the end. WIP y.
* This is a bit of a pain: if you edit the configuration by hand (I
  think dynamic setting of options won't have that problem) you get
  duplicate options, one from the file and one from what you've asserted
  in the database.
* Told'ya the dynamic db is evil.

2 years agoGave some tlc to configuration, dynamic configuration printing.
YeGoblynQueenne@splinter [Tue, 9 Aug 2016 09:40:23 +0000]
Gave some tlc to configuration, dynamic configuration printing.
* Removed a bit of noise from dynamic configuration printing, really.

2 years agoStill fiddling with tuning cross validation; still WIP
YeGoblynQueenne@splinter [Tue, 9 Aug 2016 08:17:55 +0000]
Still fiddling with tuning cross validation; still WIP
* I left this version running tuning_k_fold_cross_validation/0 for gnf
  grammars overnight and it's still going at full tilt. It's at j=5,k=5
  so it's still got most of it left to do. It's insane. And it's not
  even correct.
* Look, this is not supposed to be the project log, right? Don't just
  discuss anything that comes to mind here.
* Oh, sorry.

2 years agoFiddling with cross-validation; WIP very; removed some logging.
YeGoblynQueenne@splinter [Mon, 8 Aug 2016 23:06:18 +0000]
Fiddling with cross-validation; WIP very; removed some logging.
* Commented out logging in graph_fitting module. Should cleanup
  properly.
* Cross validation is now at the point where I use it to tune
  parameters, but I don't think it's properly nested and all yet.

2 years agoAdded printing, options for k-fold cross-validation.
YeGoblynQueenne@splinter [Mon, 8 Aug 2016 14:15:37 +0000]
Added printing, options for k-fold cross-validation.

2 years agoAdded grammar-editing option and showed grammar_printing some love.
YeGoblynQueenne@splinter [Mon, 8 Aug 2016 13:41:44 +0000]
Added grammar-editing option and showed grammar_printing some love.
* New config option edit_output_grammar/1 tells grammar_printing module
  predicates whether to open a newly learned grammar in the Swi IDE or
  not.
* Also refactored print_grammar/[1/0] a bit so that the 0-arity version
  now calls the 1-arity one, reducing copy/pasta.

2 years agoAdded more evaluation preds than you can shake a stick at. Very WIP.
YeGoblynQueenne@splinter [Mon, 8 Aug 2016 13:29:54 +0000]
Added more evaluation preds than you can shake a stick at. Very WIP.
* Including k-fold cross-validation (but no parameter tuning so far).

2 years agoAdded a few more data-sampling/splitting predicates.
YeGoblynQueenne@splinter [Mon, 8 Aug 2016 13:28:43 +0000]
Added a few more data-sampling/splitting predicates.
* Those are proliferating dangerously. I should probably make a bit of
  effort to generalise them.

2 years agoPredicate to compute average of a list of numbers.
YeGoblynQueenne@splinter [Mon, 8 Aug 2016 13:26:52 +0000]
Predicate to compute average of a list of numbers.

2 years agoPredicate to count unique sentences in corpus.
YeGoblynQueenne@splinter [Mon, 8 Aug 2016 13:26:16 +0000]
Predicate to count unique sentences in corpus.

2 years agoSlight change to logging in grammar evaluation.
YeGoblynQueenne@splinter [Mon, 8 Aug 2016 08:26:17 +0000]
Slight change to logging in grammar evaluation.

* Preparing to branch off to keep abortive attempt to do training/test
  set frequency counts in corpus utilities. Should probably not do it
  that way (by printing to a file, that is).

2 years agoAdded predicate to compute the size of training and test sets.
YeGoblynQueenne@splinter [Mon, 8 Aug 2016 06:50:10 +0000]
Added predicate to compute the size of training and test sets.

2 years agoAdded training_and_test_sets/2 to split examples in duh.
YeGoblynQueenne@splinter [Sun, 7 Aug 2016 21:26:37 +0000]
Added training_and_test_sets/2 to split examples in duh.
* Also used this in various places, but not grammar_evaluation module
  yet.
* Also added new configuration option, training_set_size/1 to set that.

2 years agoAdded splits/4 predicate to split data set to training, test sets.
YeGoblynQueenne@splinter [Sun, 7 Aug 2016 20:37:50 +0000]
Added splits/4 predicate to split data set to training, test sets.

2 years agoRemoved old grammar printing module.
YeGoblynQueenne@splinter [Sun, 7 Aug 2016 19:58:30 +0000]
Removed old grammar printing module.
* Shhhh. It's in the backups folder!
* Packrat.

2 years agoReplaced more naked calls to language start symbol by utilities wrapper.
YeGoblynQueenne@splinter [Sun, 7 Aug 2016 19:57:19 +0000]
Replaced more naked calls to language start symbol by utilities wrapper.

2 years agoReplaced naked calls to language start symbol with utilities wrapper.
YeGoblynQueenne@splinter [Sun, 7 Aug 2016 19:49:26 +0000]
Replaced naked calls to language start symbol with utilities wrapper.

2 years agoCommited configuration module by mistake.
YeGoblynQueenne@splinter [Sun, 7 Aug 2016 19:41:51 +0000]
Commited configuration module by mistake.

2 years agoRenamed training_set --> examples_set.
YeGoblynQueenne@splinter [Sun, 7 Aug 2016 19:40:23 +0000]
Renamed training_set --> examples_set.
* 'Cause we need to have training/test set, right

2 years agoAdded some logging of examples parsed within time limit.
YeGoblynQueenne@splinter [Sun, 7 Aug 2016 19:28:49 +0000]
Added some logging of examples parsed within time limit.
* This is in stochastic_grammar_evaluation/0.

2 years agoAdded directive to increase stack size in project load file.
YeGoblynQueenne@splinter [Sun, 7 Aug 2016 14:12:44 +0000]
Added directive to increase stack size in project load file.
* Seems the whole M:tG corpus breaks the stack almost always, certainly
  when training a GNF grammar.

2 years agoAdded time limit to stochastic parsing evaluation.
YeGoblynQueenne@splinter [Sun, 7 Aug 2016 13:47:20 +0000]
Added time limit to stochastic parsing evaluation.
* This is a configuration option that says how much time a stochastic
  grammar evaluation predicate (well, just the one we got right now)
  will search for a parse of a given sentence before giving up. Giving
  up means the probability of a sentence is bound to an atom
  "time_limit_exceeded", the same as returned by call_with_time_limit/2
  as an exception if the set time limit is exceeded (but currently the
  binding is by hand). It's a bit of a mess because we throw and catch a
  time_limit_exceeded exception which is slow and so on, but the point
  is to avoid involving such parses to the evaluation of the grammar. So
  it's slightly cheating but only slightly. Because parsing some rules
  can take a long time, specifically, exponential to the length of a
  string. If a string is ungrammatical, parsing normally fails quickly
  but if you get unlucky you may end up backtracking over and over again
  over and over the same string.

2 years agoAdded tandem/3 utility predicate to go over two lists in tandem.
YeGoblynQueenne@splinter [Sat, 6 Aug 2016 17:32:33 +0000]
Added tandem/3 utility predicate to go over two lists in tandem.
* This is like member/2, but takes two lists and binds the elements at
  the same position in each list to a key-value pair term. In this it's
  kind of more like select/4 but it doesn't expect the two lists to have
  the same terms in the same position.

2 years agoAdded load_sentence_frequencies/1 to corpus_utilities.
YeGoblynQueenne@splinter [Sat, 6 Aug 2016 12:11:11 +0000]
Added load_sentence_frequencies/1 to corpus_utilities.
* This will load a sentence frequencies file, or create one with a call
  to print_sentence_frequencies/0 if it doesn't exist.
* Also used this new predicate instead in stochastic grammar evaluation,
  instead of printing a nde frequencies file each time.
* Also also made (stochastic_)grammar_module/1 throw a more contextual
  exception when the currently configured grammar module doesn't exist.
  May need to make that exception a bit better.

2 years agoAdded metrics module to project load file edit section.
YeGoblynQueenne@splinter [Sat, 6 Aug 2016 09:06:47 +0000]
Added metrics module to project load file edit section.
* Also commented-out directives to edit graph_fitting and
  skeleton_transformation, since I'll be working on metrics for a wee
  bit. They do still need work done.

2 years agoAdded string-version of pdcg_derivation/5 to query interface.
YeGoblynQueenne@splinter [Sat, 6 Aug 2016 09:02:21 +0000]
Added string-version of pdcg_derivation/5 to query interface.
* Originally created in grammar evaluation where it's not that needed.
  Should see if I can use it to replace current definitions of
  pdcg_derivation/5.

2 years agoAdded stochastic grammar evaluation (using metrics module).
YeGoblynQueenne@splinter [Sat, 6 Aug 2016 09:00:18 +0000]
Added stochastic grammar evaluation (using metrics module).
* Also a bit of cleanup in grammar_evaluation module: removed explicit
  imports from libs, now that we're using rather more of them and it
  becomes a bit tedious having a long list of them.

2 years agoAdded metrics module, starting with Kullback-Leibler Divergence.
YeGoblynQueenne@splinter [Sat, 6 Aug 2016 08:49:41 +0000]
Added metrics module, starting with Kullback-Leibler Divergence.

2 years agoAdded sentence statistics.
YeGoblynQueenne@splinter [Fri, 5 Aug 2016 18:46:04 +0000]
Added sentence statistics.
* Noticed it's actually faster to do all this with library(aggregate)
  predicates- make it so.
* Also, there's quite a few stuff in corpus_utilities that can be
  abstracted and generalised so make that so too.

2 years agoAdded printing of n-grams, comments (no, just n-grams).
YeGoblynQueenne@splinter [Fri, 5 Aug 2016 15:28:45 +0000]
Added printing of n-grams, comments (no, just n-grams).

2 years agoMore fixes (hopefully) to n-gram probabilities calculation.
YeGoblynQueenne@splinter [Fri, 5 Aug 2016 14:04:09 +0000]
More fixes (hopefully) to n-gram probabilities calculation.

2 years agoCorrected (probably) n-gram model probabilities computation.
YeGoblynQueenne@splinter [Fri, 5 Aug 2016 11:07:15 +0000]
Corrected (probably) n-gram model probabilities computation.

2 years agoN-gram counts to use for model evaluation; not correct and WIP.
YeGoblynQueenne@splinter [Thu, 4 Aug 2016 19:21:04 +0000]
N-gram counts to use for model evaluation; not correct and WIP.

2 years agoAdded test data generation from maths course, with Rebber grammar stuff.
YeGoblynQueenne@splinter [Thu, 4 Aug 2016 15:37:23 +0000]
Added test data generation from maths course, with Rebber grammar stuff.
* Includes predicates to generate strings from Rebber, embedded Rebber
  grammars and also permutations of relations. Was used in the maths
  course to train LSTM models so includes some stuff that won't be
  sorely needed here. Anyway it's a bit WIP and needs some tlc.

2 years agoAdded weighted graph printing; needs tlc; they can't be loaded back.
YeGoblynQueenne@splinter [Thu, 4 Aug 2016 15:36:21 +0000]
Added weighted graph printing; needs tlc; they can't be loaded back.

2 years agoTiny fix to dynamic_configuration/0.
YeGoblynQueenne@splinter [Mon, 1 Aug 2016 20:23:50 +0000]
Tiny fix to dynamic_configuration/0.
* Forgot to print out production arities for stochastic gnf also (only
  did for deterministic one).

2 years agoSquished a little bug in sentence_completion/[2,4] and list_length/2.
YeGoblynQueenne@splinter [Mon, 1 Aug 2016 14:24:39 +0000]
Squished a little bug in sentence_completion/[2,4] and list_length/2.
* This was caused by the silly way I was using partial_sentence/2 to
  append a list of a given length to a possibly partially instantiated
  sentence. partial_sentence/2 is gone since what it really needs to do
  is what list_length/2 does anyway and list_length/2 is now used in
  sentence_completion/4 in its place.
* Also refactored list_length/2 to fix the min/1 clause which was
  previously borked (it always started at the specified length plus 1)
  and added some tests to make sure it works OK. I think I even found a
  bit of a problem with the max/1 clause.

2 years agoActually added mtg_trivial corpus, ommited in previous commit.
YeGoblynQueenne@splinter [Sun, 31 Jul 2016 16:19:44 +0000]
Actually added mtg_trivial corpus, ommited in previous commit.
* Omitted because the corpus directory is ignored- to avoid adding big
  huge files that shouldn't be carried around repositories.

2 years agoDeleted older attempts - supergrammar and learning_productions.
YeGoblynQueenne@splinter [Sun, 31 Jul 2016 15:58:12 +0000]
Deleted older attempts - supergrammar and learning_productions.

2 years agoMerge branch 'probabilities' old_versions
YeGoblynQueenne@splinter [Sun, 31 Jul 2016 15:40:00 +0000]
Merge branch 'probabilities'

2 years agoAdded small note. graph_fitting needs some tlc. probabilities
YeGoblynQueenne@splinter [Sun, 31 Jul 2016 15:38:56 +0000]
Added small note. graph_fitting needs some tlc.

2 years agoAdded another short mtg examples file.
YeGoblynQueenne@splinter [Sun, 31 Jul 2016 15:35:35 +0000]
Added another short mtg examples file.
* This one has just a handful of abilities with a bit of vartiety,
  unlike the handsim dataset.

2 years agoSmall tweak to dynamic_configuration/0 to unclutter listing.
YeGoblynQueenne@splinter [Fri, 29 Jul 2016 10:42:23 +0000]
Small tweak to dynamic_configuration/0 to unclutter listing.
* dynamic_configuration/0 will now only list production_arity/3 value
  when transformation_format/1 is set to gnf. In right-regular grammars
  production_arity is always 1 for nonterminals and determined by
  graph_arity/1 for terminals.

2 years agoAdded lexicalisation suffix to grammar output file names.
YeGoblynQueenne@splinter [Fri, 29 Jul 2016 10:27:12 +0000]
Added lexicalisation suffix to grammar output file names.

2 years agoAdded bags directory for bootstrapped grammars.
YeGoblynQueenne@splinter [Mon, 18 Jul 2016 20:35:51 +0000]
Added bags directory for bootstrapped grammars.

2 years agoTweaked bootstrap grammar & bags naming format.
YeGoblynQueenne@splinter [Sun, 17 Jul 2016 22:04:29 +0000]
Tweaked bootstrap grammar & bags naming format.

2 years agoRefactored sample/5: split to set_sample/5, sample/4.
YeGoblynQueenne@splinter [Sun, 17 Jul 2016 14:42:53 +0000]
Refactored sample/5: split to set_sample/5, sample/4.
* set_sample/5 expects a set and gives a sample and a set remainder (the
  set remainder of the input and the sample). sample/4 instead makes no
  assumption about ordering or uniqueness and does not construct a
  remainder.
* There's a tradeoff: set_sample/5 needs to ensure that the sample and
  the remainder are in the same order as in the input. This means we
  need to do some list reversing at the end of processing. This happens
  in one of the business-end predicates (sample_/5) and at that point it
  would be faster to just sort the results, but then we wouldn't be able
  to use sample_/5 with sample/4, since sample/4 must preserve
  duplicates and ordering of the data. Expect some optimisation around
  that in the next commit.

2 years agoAdded start symbol arity config option.
YeGoblynQueenne@splinter [Sat, 16 Jul 2016 21:52:52 +0000]
Added start symbol arity config option.
* This is a rule so it's not meant to be modified on-the-fly, or at all
  even. Rules are basically reporting config options, right? This one's
  used to print the start symbol of grammars with the correct arity so I
  don't have to keep fixing it byhand, stupidly.

2 years agoStopped sorting examples in examples_corpus/1.
YeGoblynQueenne@splinter [Sat, 16 Jul 2016 20:20:57 +0000]
Stopped sorting examples in examples_corpus/1.
* I'm pretty sure this gives us better probabilities.

2 years agoAdded slimmed-down names to config; made new output file name config.
YeGoblynQueenne@splinter [Sat, 16 Jul 2016 14:43:48 +0000]
Added slimmed-down names to config; made new output file name config.
* New file naming predicate includes graph and production arities, which
  are basically algorithm hyperparams and are useful to have, basically
  because they make it much easier to know what the difference is
  between two grammar files.

2 years agoSlimmed-down corpus file names. Like, at long last.
YeGoblynQueenne@splinter [Sat, 16 Jul 2016 14:41:35 +0000]
Slimmed-down corpus file names. Like, at long last.

2 years agoFixed bug in let_configuration_options/2
YeGoblynQueenne@splinter [Thu, 14 Jul 2016 21:53:19 +0000]
Fixed bug in let_configuration_options/2
* That messed up changing of mult-clause options on the fly. Should work
  better now (but still keep an eye out 'cause today I'm knackered).

2 years agoSome maintenance; also, added Lovecraft corpus stuff (but not itself)
YeGoblynQueenne@splinter [Thu, 14 Jul 2016 21:20:50 +0000]
Some maintenance; also, added Lovecraft corpus stuff (but not itself)
* Refactored config option output_file_name/2 to include graph and
  production arities.
* Printing some information at loadup, mainly dynamic_configuration/0
  output.
* Configured corpus_utilities to tokenise lovecraft corpus; might want
  to give that a config file all of its own. Or not?

2 years agoFixed bug in skeleton transformation (production/8).
YeGoblynQueenne@splinter [Thu, 14 Jul 2016 21:18:20 +0000]
Fixed bug in skeleton transformation (production/8).
* This one stopped me from making a stochastic gnf with a certain arity
  (3 in this case, but it's corpus-depednent) because it nested
  nonterminals inside trees, like so: ((nt1, nt2),nt3). This failed when
  it hit uneev in weighted_productions/3 (I think). The bug was in
  production/8. Er, that.
* Oh and- I got stochastic gnfs alright now.

2 years agoGot stochastic gnf working; WIP but working
YeGoblynQueenne@splinter [Wed, 13 Jul 2016 22:20:04 +0000]
Got stochastic gnf working; WIP but working

2 years agoMerge branch 'reconfigure' into probabilities
YeGoblynQueenne@splinter [Wed, 13 Jul 2016 08:43:54 +0000]
Merge branch 'reconfigure' into probabilities

2 years agoMade cnf_split_divisor/1 static as foretold in the prophecies of old.
YeGoblynQueenne@splinter [Wed, 13 Jul 2016 08:43:17 +0000]
Made cnf_split_divisor/1 static as foretold in the prophecies of old.

2 years agoClarified config option split_divisor/1 as cnf_split_divisor/1.
YeGoblynQueenne@splinter [Wed, 13 Jul 2016 08:41:27 +0000]
Clarified config option split_divisor/1 as cnf_split_divisor/1.
* This is used in CNF induction, which doesn't really work currently
  anyway, but I'm keeping it in for now, although might change it to
  static (so it's not reported by dynamic_configuration/0) in next
  commit.

2 years agoAdded dynamic_configuration/0 to report options modifieable on-the-fly
YeGoblynQueenne@splinter [Wed, 13 Jul 2016 08:38:40 +0000]
Added dynamic_configuration/0 to report options modifieable on-the-fly

2 years agoMerge branch 'reconfigure' into probabilities
YeGoblynQueenne@splinter [Wed, 13 Jul 2016 07:05:31 +0000]
Merge branch 'reconfigure' into probabilities

2 years agoCleanups - added comments, moved bits around etc.
YeGoblynQueenne@splinter [Wed, 13 Jul 2016 06:59:30 +0000]
Cleanups - added comments, moved bits around etc.

2 years agoRefactored graph_probabilities/2 to allow variable edge arities; WIP.
YeGoblynQueenne@splinter [Wed, 13 Jul 2016 06:30:53 +0000]
Refactored graph_probabilities/2 to allow variable edge arities; WIP.
* This is like, considerably slower than the previous version, except of
  course it allows for any graph_arity/1 value. It also uses the dynamic
  database, whereas the previous version didn't. I'm keeping the old
  version in also and probably using that when graph_arity(1) is true,
  until I find a way to make the new version as fast as the old.

2 years agoAdded some comments to let_configuration_option/[2,3].
YeGoblynQueenne@splinter [Mon, 11 Jul 2016 22:17:36 +0000]
Added some comments to let_configuration_option/[2,3].
* Also a tiny bit of clearer user prompting.

2 years agoReconfigured configuration: added reconfigure/0, traing_set module.
YeGoblynQueenne@splinter [Mon, 11 Jul 2016 21:12:30 +0000]
Reconfigured configuration: added reconfigure/0, traing_set module.
* training_set is the module where examples and language data are loaded
  into (example_string/1, start//0, terminal//0 and nontermina//0). In
  other words, everything that we previously loaded into configuration
  module and accessed through it via explicit module references. This
  was not ideal but it was the easiest thing to do, however it turned up
  a few issues now that I'm trying to do do configuration chanes
  on-the-fly. So, now there's a training_set module and it works as
  configuration module did except it only has training data, not
  configuration predicates. OK?
* OK, don't get mad at me. I just wanted to say: also removed the
  directive calling  register_world/3 from configuration module and
  added a reconfigure/0 predicate in load_configuration that does the
  same thing. This is called from the new training_set module to reload
  training data as before in configuration module. The upside of this is
  that now configuration module doesn't have any functionality in it,
  just configuration options (well, except for the configuration options
  that have a body but yeah).
* Also tested that we can change examples modules by hand (not on the
  fly yet) and found out that there's an error from grammar_evaluation
  module, that comes up when we train, evaluate, switch modules and
  train again. At that point, because we load the examples module by
  hand (and reexport its predicates) in the grammar evaluation module,
  there is a redefinition error. Sucks but it's just the evaluation
  module. I'll have to fix it at some point.

2 years agoRefactored dynamic directives to one. Why did I have two?
YeGoblynQueenne@splinter [Sun, 10 Jul 2016 20:34:55 +0000]
Refactored dynamic directives to one. Why did I have two?

2 years agoMoved register_world/3 directive to load_configuration
YeGoblynQueenne@splinter [Sun, 10 Jul 2016 20:29:43 +0000]
Moved register_world/3 directive to load_configuration
* Not sure what it was doing there. Oh, actually- I do. Having it there
  sort of took care of not having to load configuration and
  load_configuration in a specific order, but I think this horse has
  bolted and anyway moving it to load_configuration makes it even less
  easy for the user to mess up by removing it inadvertently. There's a
  new directive in configuration of course, that calls to a new
  predicate in load_configuration that does exactly the same thing but
  it's a single line (in configuration) so it makes it less easy to mess
  up. I hope. I think. Maybe.

2 years agoHid compression grammar, compression corpus stuff. Need fixing first.
YeGoblynQueenne@splinter [Sun, 10 Jul 2016 20:22:12 +0000]
Hid compression grammar, compression corpus stuff. Need fixing first.
* Specifically need to make sure they work with the new graph
  transformation bits. I think there's also a bit of fiddling to do with
  sanitised graph vertices.

2 years agoconfiguration module is now imported only from project load file.
YeGoblynQueenne@splinter [Sun, 10 Jul 2016 19:34:22 +0000]
configuration module is now imported only from project load file.
* This unifies the way configuration options are accessed from any
  module. Previously some modules accessed config options by explicitly
  referring to the configuration module, while some also called config
  option predicates without the module qualifier but imported the
  configuration module directly. So I'm making sure everything follows
  the same format from now on.
* To be fair this is a bit fiddly and it depends a lot on the order by
  which project files are loaded, but it's workable so.

* Yeah yeah. Forgot to commit the actual files. Mumble mumble.

2 years agoImplemented dynamic configuration options; WIP and untested.
YeGoblynQueenne@splinter [Sun, 10 Jul 2016 04:07:02 +0000]
Implemented dynamic configuration options; WIP and untested.

2 years agoRemoved vestigial config options; cleaned up.
YeGoblynQueenne@splinter [Sun, 10 Jul 2016 00:32:32 +0000]
Removed vestigial config options; cleaned up.
* Note well that cleanup includes:
a) predicates in graph_fitting module that called out to one of the
   vestigial options removed, specifically sort_graph/2 that never
   sorted the graph, because we 're now leaving that up to
   transformation rules to manage.
b) Some attempts at GNF version 1 that didn't quite go as planned.
   Hopefully this won't end like Resident Evil begins.