Page 1 of 2

Collation format

Posted: November 10th, 2018, 11:08 am
by Alan Bunning
If you have any interest in being able to download the CNTR transcriptions in some type of collation format (see https://greekcntr.org/manuscripts.htm), you ought to take a look at some of the ideas being proposed at https://greekcntr.org/slots.pdf. We are trying to come up with a standardized data format for universal data exchange between programs to handle all variants readings found in all Greek texts. I would be interested in getting your feedback as the issue is currently being discussed by several scholars and your input could heavily influence the direction the process takes. Thanks.

Re: Collation format

Posted: November 10th, 2018, 4:22 pm
by Jonathan Robie
Hi Alan,

Yes, something like this is definitely needed, and there has been a lot of discussion of the need for something like this in the Copenhagen Alliance. Let's talk about the best way to discuss this at the Copenhagen Alliance event at SBL.

Re: Collation format

Posted: November 12th, 2018, 5:19 am
by Jacob Rhoden
I am quite sure I am missing something here because my first question is, "why not allow greek letters to be used for Slot ID's and allow a variable length slot id?" (If the concern were about storage space, we wouldn't use base 58, but something more compact.)

i.e. The Slot ID for ονειδισωσιν ονιδισωσιν could just be either ονειδισωσιν or ονιδισωσιν.

Secondly, will there never be cases where it is ambiguous as to which "Slot ID" a particular word should be allocated to?

Re: Collation format

Posted: November 12th, 2018, 10:01 am
by Alan Bunning
Jacob Rhoden wrote: November 12th, 2018, 5:19 am I am quite sure I am missing something here because my first question is, "why not allow greek letters to be used for Slot ID's and allow a variable length slot id?" (If the concern were about storage space, we wouldn't use base 58, but something more compact.)

i.e. The Slot ID for ονειδισωσιν ονιδισωσιν could just be either ονειδισωσιν or ονιδισωσιν.
I had thought about that too early on. But the slot ID is tied to is relative position as well for comparison purposes. For example, you could have the same word appearing 5 times in the same verse and you have to know which position each is in. Some texts may only have the word 3 or 4 times, so it is important to know the relative word order and which word they are aligning to. Just knowing the words basically gives you a list of the words used in the NT, but does not help at all for alignment.
Jacob Rhoden wrote: November 12th, 2018, 5:19 am Secondly, will there never be cases where it is ambiguous as to which "Slot ID" a particular word should be allocated to?
Not in any significant way. With a fragmentary text, for example, you might have a fragment where the last visible word is και and there could be a variant there with “και blah blah blah και blah...”, with “και blah blah blah” being a variant reading. And so you are not sure if it should align with the first και or the second και. And assuming that you can tell nothing from line length, it would then be editor’s choice or perhaps you could base it on whether it follows a more Alexandrian or Byzantine text. Such cases are few and far between and don’t normally involve any words of substance.

Re: Collation format

Posted: November 12th, 2018, 10:47 am
by Jonathan Robie
Alan Bunning wrote: November 12th, 2018, 10:01 am
Jacob Rhoden wrote: November 12th, 2018, 5:19 am Secondly, will there never be cases where it is ambiguous as to which "Slot ID" a particular word should be allocated to?
Not in any significant way. With a fragmentary text, for example, you might have a fragment where the last visible word is και and there could be a variant there with “και blah blah blah και blah...”, with “και blah blah blah” being a variant reading. And so you are not sure if it should align with the first και or the second και. And assuming that you can tell nothing from line length, it would then be editor’s choice or perhaps you could base it on whether it follows a more Alexandrian or Byzantine text. Such cases are few and far between and don’t normally involve any words of substance.
If I understand correctly, this document is about labeling slots that have been identified by some other process, did I get that right?

I think the scenario looks like this: Someone like Alan prepares a database with slots for all known texts using some algorithm that aligns the texts, perhaps with human intervention. Each slot needs an identifier.

Eventually, someone finds a new manuscript that requires a new slot, which needs a new identifier. Using GUIDs means you can always create a new one. Using a short GUID probably requires some thinking to make sure that a new GUID is unique. Or perhaps you simply discard it if it is not unique.

Re: Collation format

Posted: November 12th, 2018, 10:52 am
by Alan Bunning
Jonathan Robie wrote: November 12th, 2018, 10:47 am If I understand correctly, this document is about labeling slots that have been identified by some other process, did I get that right?
I think the scenario looks like this: Someone like Alan prepares a database with slots for all known texts using some algorithm that aligns the texts, perhaps with human intervention. Each slot needs an identifier.
It was originally generated by an algorithm and is now being adjusted by hand. The same algorithm is used to align a new text that is found and identify any new variants, but then that may still need to be adjusted by hand.
Jonathan Robie wrote: November 12th, 2018, 10:47 am Eventually, someone finds a new manuscript that requires a new slot, which needs a new identifier. Using GUIDs means you can always create a new one. Using a short GUID probably requires some thinking to make sure that a new GUID is unique. Or perhaps you simply discard it if it is not unique.
Using the 3-digit slot ID's proposed would allow about 50000 spare ID's for this purpose in the NT. Using 4-digit slot ID's, there would be plenty of extras for both the NT and OT.

Re: Collation format

Posted: November 12th, 2018, 10:54 am
by Jonathan Robie
Alan Bunning wrote: November 12th, 2018, 10:52 am
Jonathan Robie wrote: November 12th, 2018, 10:47 am Eventually, someone finds a new manuscript that requires a new slot, which needs a new identifier. Using GUIDs means you can always create a new one. Using a short GUID probably requires some thinking to make sure that a new GUID is unique. Or perhaps you simply discard it if it is not unique.
Using the 3-digit slot ID's proposed would allow about 50000 spare ID's for this purpose in the NT. Using 4-digit slot ID's, there would be plenty of extras for both the NT and OT.
OK, Alan has done that thinking. Knowing Alan, that doesn't surprise me.

Re: Collation format

Posted: November 12th, 2018, 4:17 pm
by Jacob Rhoden
Alan Bunning wrote: November 12th, 2018, 10:01 am I had thought about that too early on. But the slot ID is tied to is relative position as well for comparison purposes. For example, you could have the same word appearing 5 times in the same verse and you have to know which position each is in. Some texts may only have the word 3 or 4 times, so it is important to know the relative word order and which word they are aligning to. Just knowing the words basically gives you a list of the words used in the NT, but does not help at all for alignment.
Thanks, now I understand. In your example και has different "Slot ID's". I didn't pick this up the first time I read it.

Is there an argument for keeping the Slot ID short? I guess three characters makes it easier for human processing? I am inclined to suggest that the Slot ID should be large enough to accommodate everything in Sinaiticus/Vaticanus.

Re: Collation format

Posted: November 12th, 2018, 8:59 pm
by Alan Bunning
Jacob Rhoden wrote: November 12th, 2018, 4:17 pm ID should be large enough to accommodate everything in Sinaiticus/Vaticanus.
Those were already accounted for and there still will be about 50,000 slot ID's free for the future. I think smaller is better, but I am guessing we will probably have to go with a 4-digit ID, because people will probably want the OT to be covered as well.

Re: Collation format

Posted: November 12th, 2018, 9:50 pm
by Jonathan Robie
Alan Bunning wrote: November 12th, 2018, 8:59 pm ... because people will probably want the OT to be covered as well.
I'm a person, and I think this is true.