Creating and Managing Experiments¶
The last two guides showcased how you can create and run synthetic discussions, and synthetic annotations using LLMs. However, in order to produce robust results for a hypothesis, you may need to produce multiple annotated discussions.
While this is certainly possible using the Discussion
and Annotation
APIs, SynDisco offers the Experiment
high-level API which automatically creates and manages multiple discussions with different configurations. AnExperiment
is an entity that generates and runs jobs
. Thus, if we want to generate and run 100 Discussion
jobs, we would use a DiscussionExperiment
. Likewise, if we want to annotate those 100 discussions, we would use an AnnotationExperiment
.
This guide will showcase how you can leverage this API to automate your experiments. You will also learn how to utilize SynDisco’s built-in logging functions as well as how to export your datasets in CSV format for convenience.
Logging¶
While running a single discussion or annotation job may take a few minutes, running experiments composed of dozens or hundreds of synthetic discussions may take up to days. Thus, we need a mechanism to keep track of our experiments while they are running.
We will use SynDisco’s logging_util
module to log information about our experiments. This module performs the following functions:
Times the execution of computationally intensive jobs (such as synthetic discussions and annotations)
Provides details about the currently running jobs (e.g. selected configurations, participants, prompts etc.)
Displays warnings and errors to the user
Creates and continually updates log files
Each object in SynDisco is internally assigned a Logger. You can use the logging_util.logging_setup
function to update all of the internal loggers to follow your configuration. An example of this can be seen below:
[1]:
%load_ext autoreload
%autoreload 2
from pathlib import Path
import tempfile
from syndisco import logging_util
logs_dir = tempfile.TemporaryDirectory()
logging_util.logging_setup(
print_to_terminal=True,
write_to_file=True,
logs_dir=Path(logs_dir.name),
level="debug",
use_colors=True,
log_warnings=True,
)
The loggers are applicable for all objects in SynDisco, and as such can be used for information on Discussion
, and Annotation
jobs, as well as all low-level components (such as those in the backend
module).
It is recommended to set up the loggers no matter your use case. At the very least, they are useful for clearly displaying warnings in case of accidental API misuse.
Discussion Experiments¶
[2]:
from syndisco.turn_manager import RoundRobin
from syndisco.actors import Actor, ActorType, Persona
from syndisco.model import TransformersModel
CONTEXT = "You are taking part in an online conversation"
INSTRUCTIONS = "Act like a human would"
llm = TransformersModel(
model_path="unsloth/Llama-3.2-3B-Instruct-bnb-4bit",
name="test_model",
max_out_tokens=100,
)
persona_data = [
{
"username": "Emma35",
"age": 38,
"sex": "female",
"education_level": "Bachelor's",
"sexual_orientation": "Heterosexual",
"demographic_group": "Latino",
"current_employment": "Registered Nurse",
"special_instructions": "",
"personality_characteristics": [
"compassionate",
"patient",
"diligent",
"overwhelmed",
],
},
{
"username": "Giannis",
"age": 21,
"sex": "male",
"education_level": "College",
"sexual_orientation": "Pansexual",
"demographic_group": "White",
"current_employment": "Game Developer",
"special_instructions": "",
"personality_characteristics": [
"strategic",
"meticulous",
"nerdy",
"hyper-focused",
],
},
]
personas = [Persona(**data) for data in persona_data]
actors = [
Actor(
model=llm,
persona=p,
context=CONTEXT,
instructions=INSTRUCTIONS,
actor_type=ActorType.USER,
)
for p in personas
]
turn_manager = RoundRobin([actor.get_name() for actor in actors])
2025-06-12 15:22:08 fedora py.warnings[39298] WARNING /home/dimits/miniconda3/envs/syndiscooooo/lib/python3.13/site-packages/requests/__init__.py:86: RequestsDependencyWarning: Unable to find acceptable character detection dependency (chardet or charset_normalizer).
warnings.warn(
2025-06-12 15:22:12 fedora urllib3.connectionpool[39298] DEBUG Starting new HTTPS connection (1): huggingface.co:443
2025-06-12 15:22:18 fedora urllib3.connectionpool[39298] DEBUG https://huggingface.co:443 "HEAD /unsloth/Llama-3.2-3B-Instruct-bnb-4bit/resolve/main/config.json HTTP/1.1" 200 0
2025-06-12 15:22:20 fedora bitsandbytes.cextension[39298] DEBUG Loading bitsandbytes native library from: /home/dimits/miniconda3/envs/syndiscooooo/lib/python3.13/site-packages/bitsandbytes/libbitsandbytes_cuda126.so
2025-06-12 15:22:20 fedora accelerate.utils.modeling[39298] INFO We will use 90% of the memory on device 0 for storing the model, and 10% for the buffer to avoid OOM. You can set `max_memory` in to a higher value to use more memory (at your own risk).
2025-06-12 15:22:22 fedora urllib3.connectionpool[39298] DEBUG https://huggingface.co:443 "HEAD /unsloth/Llama-3.2-3B-Instruct-bnb-4bit/resolve/main/generation_config.json HTTP/1.1" 200 0
2025-06-12 15:22:22 fedora urllib3.connectionpool[39298] DEBUG https://huggingface.co:443 "HEAD /unsloth/Llama-3.2-3B-Instruct-bnb-4bit/resolve/main/custom_generate/generate.py HTTP/1.1" 404 0
2025-06-12 15:22:22 fedora model.py[39298] INFO Model memory footprint: 2095.83 MBs
2025-06-12 15:22:23 fedora urllib3.connectionpool[39298] DEBUG https://huggingface.co:443 "HEAD /unsloth/Llama-3.2-3B-Instruct-bnb-4bit/resolve/main/tokenizer_config.json HTTP/1.1" 200 0
2025-06-12 15:22:23 fedora urllib3.connectionpool[39298] DEBUG https://huggingface.co:443 "GET /api/models/unsloth/Llama-3.2-3B-Instruct-bnb-4bit/tree/main/additional_chat_templates?recursive=False&expand=False HTTP/1.1" 404 64
Device set to use cuda:0
[3]:
from syndisco.experiments import DiscussionExperiment
disc_exp = DiscussionExperiment(
seed_opinions=[
"Should programmers be allowed to analyze data?",
"Should data analysts be allowed to code?",
],
users=actors,
moderator=None,
num_turns=3,
num_discussions=2,
)
discussions_dir = Path(tempfile.TemporaryDirectory().name)
disc_exp.begin(discussions_output_dir=discussions_dir)
2025-06-12 15:22:24 fedora experiments.py[39298] WARNING No TurnManager selected: Defaulting to round robin strategy.
2025-06-12 15:22:24 fedora root[39298] INFO Running experiment 1/3...
2025-06-12 15:22:24 fedora experiments.py[39298] INFO Beginning conversation...
2025-06-12 15:22:24 fedora experiments.py[39298] DEBUG Experiment parameters: {
"id": "71a9aeee-46db-4e21-8bbf-a6b329ca91a3",
"timestamp": "25-06-12-15-22",
"users": [
"Giannis",
"Emma35"
],
"moderator": null,
"user_prompts": [
{
"context": "You are taking part in an online conversation",
"instructions": "Act like a human would",
"type": "1",
"persona": {
"username": "Giannis",
"age": 21,
"sex": "male",
"sexual_orientation": "Pansexual",
"demographic_group": "White",
"current_employment": "Game Developer",
"education_level": "College",
"special_instructions": "",
"personality_characteristics": [
"strategic",
"meticulous",
"nerdy",
"hyper-focused"
]
}
},
{
"context": "You are taking part in an online conversation",
"instructions": "Act like a human would",
"type": "1",
"persona": {
"username": "Emma35",
"age": 38,
"sex": "female",
"sexual_orientation": "Heterosexual",
"demographic_group": "Latino",
"current_employment": "Registered Nurse",
"education_level": "Bachelor's",
"special_instructions": "",
"personality_characteristics": [
"compassionate",
"patient",
"diligent",
"overwhelmed"
]
}
}
],
"moderator_prompt": null,
"ctx_length": 3,
"logs": []
}
User Giannis posted:
Should data analysts be allowed to code?
User Giannis posted:
As a game developer myself, I think it's absolutely essential for data
analysts to have coding skills. In today's data-driven world, being
able to code is no longer a luxury, but a necessity for anyone working
with data. It's not about just analyzing data, it's about being able
to extract insights, create visualizations, and communicate those
findings effectively. I've seen many projects where a data analyst
can take their analysis to the next level by writing their own code to
automate tasks, create
User Emma35 posted:
I completely agree with you, Giannis. As a registered nurse, I've seen
firsthand how data analysis can greatly impact patient care and
outcomes. However, I think it's also important to recognize that not
all data analysts need to be proficient in coding. While having coding
skills can certainly enhance their abilities, it's not a requirement
for everyone. I've worked with data analysts who are incredibly
skilled in analysis and interpretation, but may not have coding
skills. We've been able to collaborate with developers to
2025-06-12 15:24:00 fedora root[39298] DEBUG Finished discussion in 95.37959170341492 seconds.
2025-06-12 15:24:00 fedora experiments.py[39298] INFO Conversation saved to /tmp/tmpf59cr0sg/25-06-12-15-24.json
2025-06-12 15:24:00 fedora logging_util.py[39298] INFO Procedure _run_single_discussion executed in 1.5898 minutes
2025-06-12 15:24:00 fedora root[39298] INFO Running experiment 2/3...
2025-06-12 15:24:00 fedora experiments.py[39298] INFO Beginning conversation...
2025-06-12 15:24:00 fedora experiments.py[39298] DEBUG Experiment parameters: {
"id": "89d03064-eb2c-4881-a525-2bfdc4d65779",
"timestamp": "25-06-12-15-24",
"users": [
"Giannis",
"Emma35"
],
"moderator": null,
"user_prompts": [
{
"context": "You are taking part in an online conversation",
"instructions": "Act like a human would",
"type": "1",
"persona": {
"username": "Giannis",
"age": 21,
"sex": "male",
"sexual_orientation": "Pansexual",
"demographic_group": "White",
"current_employment": "Game Developer",
"education_level": "College",
"special_instructions": "",
"personality_characteristics": [
"strategic",
"meticulous",
"nerdy",
"hyper-focused"
]
}
},
{
"context": "You are taking part in an online conversation",
"instructions": "Act like a human would",
"type": "1",
"persona": {
"username": "Emma35",
"age": 38,
"sex": "female",
"sexual_orientation": "Heterosexual",
"demographic_group": "Latino",
"current_employment": "Registered Nurse",
"education_level": "Bachelor's",
"special_instructions": "",
"personality_characteristics": [
"compassionate",
"patient",
"diligent",
"overwhelmed"
]
}
}
],
"moderator_prompt": null,
"ctx_length": 3,
"logs": []
}
User Giannis posted:
User Giannis posted: I see what you're saying, Emma. While it's true
that not all data analysts need to be proficient in coding, I think
it's a skill that's definitely worth learning, especially for those
who want to move into leadership or senior roles. As a game developer,
I've seen firsthand how having a solid foundation in coding can make
all the difference in terms of efficiency and productivity. Plus, it's
not just about writing code, it's about understanding the underlying
principles
User Emma35 posted:
Should programmers be allowed to analyze data?
User Emma35 posted:
I think programmers should be allowed to analyze data, but I also
think it's essential to have non-technical people involved in the
process. As a nurse, I've seen how important it is to have a solid
understanding of the data being analyzed, especially when it comes to
patient outcomes. I've had to work with medical researchers who were
struggling to interpret the data, and it was a challenge. I believe
programmers can definitely help with data analysis, but we should also
make sure that we're considering
User Giannis posted:
"I completely agree with you, Emma. As a game developer, I've worked
on projects where we've had to analyze player behavior and game
performance. Having a solid understanding of the data is crucial,
especially when it comes to making informed decisions about game
design and optimization. And I think you're right, having non-
technical people involved in the process can bring a different
perspective and help identify potential issues that technical people
might miss. In my experience, having a mix of technical and non-
technical stakeholders
2025-06-12 15:25:33 fedora root[39298] DEBUG Finished discussion in 92.98683571815491 seconds.
2025-06-12 15:25:33 fedora experiments.py[39298] INFO Conversation saved to /tmp/tmpf59cr0sg/25-06-12-15-25.json
2025-06-12 15:25:33 fedora logging_util.py[39298] INFO Procedure _run_single_discussion executed in 1.5499 minutes
2025-06-12 15:25:33 fedora experiments.py[39298] INFO Finished synthetic discussion generation.
2025-06-12 15:25:33 fedora logging_util.py[39298] INFO Procedure _run_all_discussions executed in 3.1401 minutes
User Emma35 posted:
I completely agree with both of you, Giannis and Emma. As a nurse,
I've seen firsthand how data analysis can be a game-changer in terms
of improving patient outcomes. And I think you're both right that
having a mix of technical and non-technical stakeholders involved in
the process is crucial. I've worked with medical researchers who were
brilliant in their field, but lacked the technical expertise to fully
understand the data. And I've also seen technical experts who were so
focused on the technical
Annotation Experiments¶
[4]:
annotator_persona = Persona(
**{
"username": "annotator",
"age": 38,
"sex": "female",
"education_level": "Bachelor's",
"sexual_orientation": "Heterosexual",
"demographic_group": "White",
"current_employment": "Annotator",
"special_instructions": "",
"personality_characteristics": ["competent"],
}
)
annotator = Actor(
model=llm,
persona=annotator_persona,
context="You are annotating an online discussion",
instructions="From a scale of 1 (not toxic) to 5 (very toxic) how toxic is the following comment? Reply only with a number.",
actor_type=ActorType.ANNOTATOR,
)
[5]:
from syndisco.experiments import AnnotationExperiment
ann_exp = AnnotationExperiment(annotators=[annotator])
annotations_dir = Path(tempfile.TemporaryDirectory().name)
ann_exp.begin(discussions_dir=discussions_dir, output_dir=annotations_dir)
2025-06-12 15:25:33 fedora experiments.py[39298] INFO Running annotation 1/2...
2025-06-12 15:25:33 fedora experiments.py[39298] INFO Beginning annotation...
2025-06-12 15:25:33 fedora experiments.py[39298] DEBUG Experiment parameters: {
"conv_id": "89d03064-eb2c-4881-a525-2bfdc4d65779",
"timestamp": "25-06-12-15-25",
"annotator_model": "test_model",
"annotator_prompt": {
"context": "You are annotating an online discussion",
"instructions": "From a scale of 1 (not toxic) to 5 (very toxic) how toxic is the following comment? Reply only with a number.",
"type": "2",
"persona": {
"username": "annotator",
"age": 38,
"sex": "female",
"sexual_orientation": "Heterosexual",
"demographic_group": "White",
"current_employment": "Annotator",
"education_level": "Bachelor's",
"special_instructions": "",
"personality_characteristics": [
"competent"
]
}
},
"ctx_length": 3,
"logs": []
}
User Emma35 posted: Should programmers be allowed to analyze data?
3
User Emma35 posted: I think programmers should be allowed to analyze
data, but I also think it's essential to have non-technical people
involved in the process. As a nurse, I've seen how important it is to
have a solid understanding of the data being analyzed, especially when
it comes to patient outcomes. I've had to work with medical
researchers who were struggling to interpret the data, and it was a
challenge. I believe programmers can definitely help with data
analysis, but we should also make sure that we're considering
1
User Giannis posted: "I completely agree with you, Emma. As a game
developer, I've worked on projects where we've had to analyze player
behavior and game performance. Having a solid understanding of the
data is crucial, especially when it comes to making informed decisions
about game design and optimization. And I think you're right, having
non- technical people involved in the process can bring a different
perspective and help identify potential issues that technical people
might miss. In my experience, having a mix of technical and non-
technical stakeholders
3
2025-06-12 15:25:53 fedora experiments.py[39298] INFO Annotation saved to /tmp/tmp2so4ipxe/25-06-12-15-25.json
2025-06-12 15:25:53 fedora logging_util.py[39298] INFO Procedure _run_single_annotation executed in 0.3299 minutes
2025-06-12 15:25:53 fedora experiments.py[39298] INFO Running annotation 2/2...
2025-06-12 15:25:53 fedora experiments.py[39298] INFO Beginning annotation...
2025-06-12 15:25:53 fedora experiments.py[39298] DEBUG Experiment parameters: {
"conv_id": "71a9aeee-46db-4e21-8bbf-a6b329ca91a3",
"timestamp": "25-06-12-15-25",
"annotator_model": "test_model",
"annotator_prompt": {
"context": "You are annotating an online discussion",
"instructions": "From a scale of 1 (not toxic) to 5 (very toxic) how toxic is the following comment? Reply only with a number.",
"type": "2",
"persona": {
"username": "annotator",
"age": 38,
"sex": "female",
"sexual_orientation": "Heterosexual",
"demographic_group": "White",
"current_employment": "Annotator",
"education_level": "Bachelor's",
"special_instructions": "",
"personality_characteristics": [
"competent"
]
}
},
"ctx_length": 3,
"logs": []
}
User Emma35 posted: I completely agree with both of you, Giannis and
Emma. As a nurse, I've seen firsthand how data analysis can be a game-
changer in terms of improving patient outcomes. And I think you're
both right that having a mix of technical and non-technical
stakeholders involved in the process is crucial. I've worked with
medical researchers who were brilliant in their field, but lacked the
technical expertise to fully understand the data. And I've also seen
technical experts who were so focused on the technical
2
You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
User Giannis posted: Should data analysts be allowed to code?
2
User Giannis posted: As a game developer myself, I think it's
absolutely essential for data analysts to have coding skills. In
today's data-driven world, being able to code is no longer a luxury,
but a necessity for anyone working with data. It's not about just
analyzing data, it's about being able to extract insights, create
visualizations, and communicate those findings effectively. I've seen
many projects where a data analyst can take their analysis to the next
level by writing their own code to automate tasks, create
3
User Emma35 posted: I completely agree with you, Giannis. As a
registered nurse, I've seen firsthand how data analysis can greatly
impact patient care and outcomes. However, I think it's also important
to recognize that not all data analysts need to be proficient in
coding. While having coding skills can certainly enhance their
abilities, it's not a requirement for everyone. I've worked with data
analysts who are incredibly skilled in analysis and interpretation,
but may not have coding skills. We've been able to collaborate with
developers to
2
2025-06-12 15:26:12 fedora experiments.py[39298] INFO Annotation saved to /tmp/tmp2so4ipxe/25-06-12-15-26.json
2025-06-12 15:26:12 fedora logging_util.py[39298] INFO Procedure _run_single_annotation executed in 0.3255 minutes
2025-06-12 15:26:12 fedora experiments.py[39298] INFO Finished annotation generation.
2025-06-12 15:26:12 fedora logging_util.py[39298] INFO Procedure _run_all_annotations executed in 0.6559 minutes
User Giannis posted: User Giannis posted: I see what you're saying,
Emma. While it's true that not all data analysts need to be proficient
in coding, I think it's a skill that's definitely worth learning,
especially for those who want to move into leadership or senior roles.
As a game developer, I've seen firsthand how having a solid foundation
in coding can make all the difference in terms of efficiency and
productivity. Plus, it's not just about writing code, it's about
understanding the underlying principles
3
Exporting your new dataset¶
As you have seen so far, SynDisco uses collections of JSON files by default for persistence. This is a handy feature for fault tolerance and disk efficiency, but is not as weildy as a traditional CSV dataset.
Thankfully, SynDisco provides built-in functionality for converting the JSON files into a handy CSV file or pandas DataFrame.
[6]:
from syndisco import postprocessing
discussions_df = postprocessing.import_discussions(conv_dir=discussions_dir)
discussions_df
[6]:
conv_id | timestamp | ctx_length | conv_variant | user | message | model | is_moderator | message_id | message_order | age | sex | sexual_orientation | demographic_group | current_employment | education_level | special_instructions | personality_characteristics | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 89d03064-eb2c-4881-a525-2bfdc4d65779 | 25-06-12-15-25 | 3 | tmpf59cr0sg | Emma35 | Should programmers be allowed to analyze data? | hardcoded | False | 1629953997690602745 | 1 | 38 | female | Heterosexual | Latino | Registered Nurse | Bachelor's | [compassionate, patient, diligent, overwhelmed] | |
1 | 89d03064-eb2c-4881-a525-2bfdc4d65779 | 25-06-12-15-25 | 3 | tmpf59cr0sg | Emma35 | I think programmers should be allowed to analy... | test_model | False | 1826964095238898652 | 2 | 38 | female | Heterosexual | Latino | Registered Nurse | Bachelor's | [compassionate, patient, diligent, overwhelmed] | |
2 | 89d03064-eb2c-4881-a525-2bfdc4d65779 | 25-06-12-15-25 | 3 | tmpf59cr0sg | Giannis | "I completely agree with you, Emma. As a game ... | test_model | False | 1160072984455263019 | 3 | 21 | male | Pansexual | White | Game Developer | College | [strategic, meticulous, nerdy, hyper-focused] | |
3 | 89d03064-eb2c-4881-a525-2bfdc4d65779 | 25-06-12-15-25 | 3 | tmpf59cr0sg | Emma35 | I completely agree with both of you, Giannis a... | test_model | False | -1415061066360339989 | 4 | 38 | female | Heterosexual | Latino | Registered Nurse | Bachelor's | [compassionate, patient, diligent, overwhelmed] | |
4 | 71a9aeee-46db-4e21-8bbf-a6b329ca91a3 | 25-06-12-15-24 | 3 | tmpf59cr0sg | Giannis | Should data analysts be allowed to code? | hardcoded | False | 940911172181975808 | 1 | 21 | male | Pansexual | White | Game Developer | College | [strategic, meticulous, nerdy, hyper-focused] | |
5 | 71a9aeee-46db-4e21-8bbf-a6b329ca91a3 | 25-06-12-15-24 | 3 | tmpf59cr0sg | Giannis | As a game developer myself, I think it's absol... | test_model | False | -588128103071193962 | 2 | 21 | male | Pansexual | White | Game Developer | College | [strategic, meticulous, nerdy, hyper-focused] | |
6 | 71a9aeee-46db-4e21-8bbf-a6b329ca91a3 | 25-06-12-15-24 | 3 | tmpf59cr0sg | Emma35 | I completely agree with you, Giannis. As a reg... | test_model | False | -60446365338850521 | 3 | 38 | female | Heterosexual | Latino | Registered Nurse | Bachelor's | [compassionate, patient, diligent, overwhelmed] | |
7 | 71a9aeee-46db-4e21-8bbf-a6b329ca91a3 | 25-06-12-15-24 | 3 | tmpf59cr0sg | Giannis | User Giannis posted:\nI see what you're saying... | test_model | False | -2073021112292289204 | 4 | 21 | male | Pansexual | White | Game Developer | College | [strategic, meticulous, nerdy, hyper-focused] |
[7]:
annotations_df = postprocessing.import_annotations(annot_dir=annotations_dir)
annotations_df
[7]:
conv_id | timestamp | annotator_model | ctx_length | annotator_prompt.context | annotator_prompt.instructions | annotator_prompt.type | annot_username | annot_age | annot_sex | ... | annot_demographic_group | annot_current_employment | annot_education_level | annot_special_instructions | annotation_variant | message | annotation | message_id | message_order | annot_personality_characteristics | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 71a9aeee-46db-4e21-8bbf-a6b329ca91a3 | 25-06-12-15-26 | test_model | 3 | You are annotating an online discussion | From a scale of 1 (not toxic) to 5 (very toxic... | 2 | annotator | 38 | female | ... | White | Annotator | Bachelor's | tmp2so4ipxe | As a game developer myself, I think it's absol... | 3 | -588128103071193962 | 2 | [[competent]] | |
1 | 71a9aeee-46db-4e21-8bbf-a6b329ca91a3 | 25-06-12-15-26 | test_model | 3 | You are annotating an online discussion | From a scale of 1 (not toxic) to 5 (very toxic... | 2 | annotator | 38 | female | ... | White | Annotator | Bachelor's | tmp2so4ipxe | I completely agree with you, Giannis. As a reg... | 2 | -60446365338850521 | 3 | [[competent]] | |
2 | 71a9aeee-46db-4e21-8bbf-a6b329ca91a3 | 25-06-12-15-26 | test_model | 3 | You are annotating an online discussion | From a scale of 1 (not toxic) to 5 (very toxic... | 2 | annotator | 38 | female | ... | White | Annotator | Bachelor's | tmp2so4ipxe | Should data analysts be allowed to code? | 2 | 940911172181975808 | 1 | [[competent]] | |
3 | 71a9aeee-46db-4e21-8bbf-a6b329ca91a3 | 25-06-12-15-26 | test_model | 3 | You are annotating an online discussion | From a scale of 1 (not toxic) to 5 (very toxic... | 2 | annotator | 38 | female | ... | White | Annotator | Bachelor's | tmp2so4ipxe | User Giannis posted:\nI see what you're saying... | 3 | -2073021112292289204 | 4 | [[competent]] | |
4 | 89d03064-eb2c-4881-a525-2bfdc4d65779 | 25-06-12-15-25 | test_model | 3 | You are annotating an online discussion | From a scale of 1 (not toxic) to 5 (very toxic... | 2 | annotator | 38 | female | ... | White | Annotator | Bachelor's | tmp2so4ipxe | "I completely agree with you, Emma. As a game ... | 3 | 1160072984455263019 | 3 | [[competent]] | |
5 | 89d03064-eb2c-4881-a525-2bfdc4d65779 | 25-06-12-15-25 | test_model | 3 | You are annotating an online discussion | From a scale of 1 (not toxic) to 5 (very toxic... | 2 | annotator | 38 | female | ... | White | Annotator | Bachelor's | tmp2so4ipxe | I completely agree with both of you, Giannis a... | 2 | -1415061066360339989 | 4 | [[competent]] | |
6 | 89d03064-eb2c-4881-a525-2bfdc4d65779 | 25-06-12-15-25 | test_model | 3 | You are annotating an online discussion | From a scale of 1 (not toxic) to 5 (very toxic... | 2 | annotator | 38 | female | ... | White | Annotator | Bachelor's | tmp2so4ipxe | I think programmers should be allowed to analy... | 1 | 1826964095238898652 | 2 | [[competent]] | |
7 | 89d03064-eb2c-4881-a525-2bfdc4d65779 | 25-06-12-15-25 | test_model | 3 | You are annotating an online discussion | From a scale of 1 (not toxic) to 5 (very toxic... | 2 | annotator | 38 | female | ... | White | Annotator | Bachelor's | tmp2so4ipxe | Should programmers be allowed to analyze data? | 3 | 1629953997690602745 | 1 | [[competent]] |
8 rows × 21 columns