Statistical Restoration

Tell us about interesting projects involving biblical Greek. Collaborative projects involving biblical Greek may use this forum for their communication - please contact jonathan.robie@ibiblio.org if you want to use this forum for your project.
Alan Bunning
Posts: 299
Joined: June 5th, 2011, 7:31 am
Contact:

Statistical Restoration

Post by Alan Bunning »

It has been many years in the making, but the CNTR is now pleased to announce the beta version of the first computer-generated Greek New Testament, which for now I am going to call the “Statistical Restoration” (SR). The creation of the SR text offers a paradigm shift in the field of textual criticism, replacing the subjectivity of human editors with the use of objective statistical and computational methods. All the earliest manuscript evidence is fed directly to a computer program as raw data, which generates a Greek New Testament to reflect the most probable text based on statistical analysis and algorithms designed to mimic the processes of textual criticism, weighing both internal and external evidence. The SR offers a number of significant advantages:

• Weighs the raw manuscript data consistently and objectively without human intervention based solely on scientific principles.
• Can be updated automatically whenever new witnesses are added.
• Automatically comes with morphological parsing, Enhanced Strong’s Numbers, English interlinear, and the Universal Apparatus.
• Will later be released under an open license for use by the global Church, after it moves from the beta stage into production.

A number of breakthroughs had to occur along the way in order to accomplish this feat, including the automatic determination of variant unit boundaries and their relationships to each other, the classification of homophones based on the orthographical-priority method, and rating the statistical reliability of manuscripts against the corpus of data. The scientific underpinnings for this are explained in six different papers which are planned to be submitted for publication, three of which have already been presented at the Society of Biblical Literature conferences. I won’t go into all of the details here, but more information about the SR text can be found in this overview video: https://greekcntr.org/resources/4SRGNT.mp4

Note that this beta version of the SR text is subject to change as it is scrutinized by others for possible flaws. This version is about 1.5% different than the Nestle-Aland 28th edition, but some them are merely orthographical differences based on how I deal with phonology. Depending on the options used, it could be made to be less than 1% different, but the goal here was not to try to replicate any particular critical text, but rather to apply a scientific-based, data-driven approach derived from the raw data. It takes about two seconds for the program to generate one Greek New Testament, and the text was optimized by running batches of 1000 texts at a time with varying parameters.

The beta version is now visible on the CNTR website in the collation view: https://greekcntr.org/collation/index.htm. Try doing a hard refresh (Ctrl-F5) if you don’t see the updates. If you would like to help, I would welcome feedback about any places where you don’t think the computer got it right, based on the dataset that it is working from. Certainly, there are plans for the dataset to be expanded in the future.
S Walch
Posts: 274
Joined: June 13th, 2011, 4:27 pm
Location: Manchester, UK

Re: Statistical Restoration

Post by S Walch »

Excellent work, Alan! Currently looking through some verses of interest, and I've come across what to me seems to be some issues, but may be a result of your Statistical analysis software and hence unintentional:

Mark 6:40 - the witnesses include either one of ἀνέπεσαν/ἀνέπεσον, but the statistical restoration (SR) includes neither, rending the sentence unintelligible without the verb.

Mark 6:43 - the witnesses include either κλάσματα/κλασμάτων, but like the above the SR includes neither. This doesn't render the sentence unintelligible, but I would expect either of the two to appear in the main text. Likely you've explained this sort of thing in one of your papers which I've yet to read.

Mark 7:24 - similar to the above, witnesses with either ἠδυνήθη/ἠδυνάσθη but SR has neither. Would say this is necessary to make sense of the sentence, and wouldn't expect it to be omitted for any reason.

Mark 7:35 - again like the above. Witnesses have either ἠνοίγησαν/διηνοίχθησαν, missing from SR, required to make sense of the sentence.

Mark 8:24 - again as the above. Witnesses have either ἔλεγεν/εἶπεν/λεγει, none are in SR, somewhat necessary to make sense of sentence.


That's all I've found at the moment. Will update here if I find anymore.
Alan Bunning
Posts: 299
Joined: June 5th, 2011, 7:31 am
Contact:

Re: Statistical Restoration

Post by Alan Bunning »

S Walch wrote: March 22nd, 2022, 10:08 am Excellent work, Alan! Currently looking through some verses of interest, and I've come across what to me seems to be some issues, but may be a result of your Statistical analysis software and hence unintentional:

Mark 6:40 - the witnesses include either one of ἀνέπεσαν/ἀνέπεσον, but the statistical restoration (SR) includes neither, rending the sentence unintelligible without the verb.

Mark 6:43 - the witnesses include either κλάσματα/κλασμάτων, but like the above the SR includes neither. This doesn't render the sentence unintelligible, but I would expect either of the two to appear in the main text. Likely you've explained this sort of thing in one of your papers which I've yet to read.

Mark 7:24 - similar to the above, witnesses with either ἠδυνήθη/ἠδυνάσθη but SR has neither. Would say this is necessary to make sense of the sentence, and wouldn't expect it to be omitted for any reason.

Mark 7:35 - again like the above. Witnesses have either ἠνοίγησαν/διηνοίχθησαν, missing from SR, required to make sense of the sentence.

Mark 8:24 - again as the above. Witnesses have either ἔλεγεν/εἶπεν/λεγει, none are in SR, somewhat necessary to make sense of sentence.


That's all I've found at the moment. Will update here if I find anymore.
Thanks for pointing those out. There is obviously a bug in the program somewhere that I am going to have to track down.

<update>Whew! The text was correct, but there was merely a problem of displaying it on the website. I will try to get that corrected shortly.
Alan Bunning
Posts: 299
Joined: June 5th, 2011, 7:31 am
Contact:

Re: Statistical Restoration

Post by Alan Bunning »

Okay, it looks like the website bug is fixed and took care of all of those problems. Thanks S Walch!
S Walch
Posts: 274
Joined: June 13th, 2011, 4:27 pm
Location: Manchester, UK

Re: Statistical Restoration

Post by S Walch »

No trouble, Alan. Glad it was something simpler than an issue with the programme itself. :)
Thom Chittom
Posts: 7
Joined: February 1st, 2013, 6:53 pm
Location: Nashville, TN
Contact:

Re: Statistical Restoration

Post by Thom Chittom »

This is a really amazing tool! The implications of its existence and the ongoing affect it will have on a half-dozen fields of research is breathtaking. Thank you for creating it and making it available.
Alan Bunning
Posts: 299
Joined: June 5th, 2011, 7:31 am
Contact:

Re: Statistical Restoration

Post by Alan Bunning »

The Center for New Testament Restoration (CNTR) is pleased to announce the official release of the Statistical Restoration (SR) which is the first computer-generated Greek New Testament. The creation of the SR text offers a paradigm shift in the field of textual criticism, replacing the subjective theological bias of human editors with the use of objective statistical and computational methods, rooted in data science and computer science. All the earliest manuscript evidence is fed directly to a computer program as raw data, which generates a Greek New Testament attempting to reflect the most probable text based on statistical analysis and algorithms designed to simulate a reasoned-eclecticism approach of textual criticism, weighing both internal and external evidence. The SR is only about 1% different from the Nestle-Aland text, but some of those are merely orthographical differences that don’t affect the meaning. The motivation, rationale, limitations, and implications for creating a text like this are described in the new book, “Restoration of the New Testament: The Case for Scientific Textual Criticism”.

Some information about the SR can be found in its Introduction and in this overview video. The more technical aspects of the SR that would interest scientists and developers will be released later in a paper that is planned to be submitted for publication. The SR is now available for download from GitHub in several different formats. The SR text is released under the Creative Commons Attribution 4.0 International License (CC-BY 4.0) and the source code for the main program is released under the GNU General Public License 3.0 (GPLv3). Providing the SR under open licenses allows others to build on the work and contribute other improvements to serve the interests of the global Church.

After being in beta for about seven months, the SR now receives its own place in the CNTR collation. Many thanks to those who provided valuable feedback to the SR while it was in beta. The SR supersedes the Bunning Heuristic Prototype (BHP), which had a successful run and served as the basis for many Bible translations all over the world, but it has finished serving its initial purpose and thus is now dropped from the collation. The new entry in the CNTR collation called “Developmental Statistics” will represent the ongoing development work toward the second edition of the SR. If any Christian scholars or developers are interested in getting in on the ground floor of that and possibly taking over that effort, please contact the CNTR.
Jonathan Robie
Posts: 4158
Joined: May 5th, 2011, 5:34 pm
Location: Durham, NC
Contact:

Re: Statistical Restoration

Post by Jonathan Robie »

Congratulations, Alan! Thanks for all the work you have devoted to this.

Is the text available separately for download?
ἐξίσταντο δὲ πάντες καὶ διηποροῦντο, ἄλλος πρὸς ἄλλον λέγοντες, τί θέλει τοῦτο εἶναι;
http://jonathanrobie.biblicalhumanities.org/
Alan Bunning
Posts: 299
Joined: June 5th, 2011, 7:31 am
Contact:

Re: Statistical Restoration

Post by Alan Bunning »

Jonathan Robie wrote: October 17th, 2022, 11:57 am Congratulations, Alan! Thanks for all the work you have devoted to this.

Is the text available separately for download?
Oops. Sorry, missed the most important link. You can download it here. https://github.com/Center-for-New-Testa ... oration/SR
Jonathan Robie
Posts: 4158
Joined: May 5th, 2011, 5:34 pm
Location: Durham, NC
Contact:

Re: Statistical Restoration

Post by Jonathan Robie »

Alan Bunning wrote: October 17th, 2022, 12:07 pm
Jonathan Robie wrote: October 17th, 2022, 11:57 am Congratulations, Alan! Thanks for all the work you have devoted to this.

Is the text available separately for download?
Oops. Sorry, missed the most important link. You can download it here. https://github.com/Center-for-New-Testa ... oration/SR
Excellent - thanks!
ἐξίσταντο δὲ πάντες καὶ διηποροῦντο, ἄλλος πρὸς ἄλλον λέγοντες, τί θέλει τοῦτο εἶναι;
http://jonathanrobie.biblicalhumanities.org/
Post Reply

Return to “Projects”