Implementing A/B Testing Simulation in Python

A/B testing is a crucial method in product development and marketing to determine which version of a feature, design, or content performs better. This challenge asks you to simulate an A/B test scenario and analyze its results programmatically in Python. You will create a system to assign users to different test groups and then calculate the conversion rates to determine the winning variant.

Problem Description

You need to build a Python class or set of functions that can simulate an A/B test. This simulation should:

1. Assign Users to Groups: Given a total number of users and the desired split ratio (e.g., 50/50, 60/40), assign each user to either "Variant A" or "Variant B".
2. Simulate Conversions: Based on predefined conversion probabilities for each variant, determine if a user "converts" or not.
3. Calculate Metrics: After simulating the test for all users, calculate the conversion rate for each variant.
4. Determine the Winner: Based on the calculated conversion rates, identify which variant is statistically "better" (i.e., has a higher conversion rate).

Key Requirements:

• A function or method to initialize and run the A/B test simulation.
• The ability to specify the total number of users.
• The ability to specify the desired split ratio between Variant A and Variant B.
• The ability to specify the probability of conversion for each variant.
• The output should clearly show the number of users in each group and their respective conversion rates.
• A clear indication of which variant is the winner.

Expected Behavior:

The simulation should generate a random assignment of users to groups based on the split ratio and then, for each user, randomly decide if they convert based on their variant's conversion probability. The final output should be a summary of the test results.

Edge Cases to Consider:

• Zero Users: What happens if 0 users are simulated?
• 100% Split for One Variant: How does the system handle assigning all users to one variant?
• Conversion Probability of 0 or 1: How are these extreme probabilities handled?

Examples

Example 1:

Input: total_users = 1000 split_ratio = {'A': 0.5, 'B': 0.5} conversion_probabilities = {'A': 0.1, 'B': 0.12}

Output:

A/B Test Results:
--------------------
Total Users: 1000

Variant A:
  Users: 500
  Conversions: [random number between 0 and 500, likely around 50]
  Conversion Rate: [e.g., 0.105]

Variant B:
  Users: 500
  Conversions: [random number between 0 and 500, likely around 60]
  Conversion Rate: [e.g., 0.122]

Winner: Variant B

Explanation: 1000 users are split evenly between A and B (500 each). For Variant A, approximately 10% convert (around 50 users), resulting in a conversion rate of ~0.105. For Variant B, approximately 12% convert (around 60 users), resulting in a conversion rate of ~0.122. Since Variant B has a higher conversion rate, it is declared the winner. (Note: Actual conversion numbers will vary due to randomness.)

Example 2:

Input: total_users = 500 split_ratio = {'A': 0.7, 'B': 0.3} conversion_probabilities = {'A': 0.05, 'B': 0.08}

Output:

A/B Test Results:
--------------------
Total Users: 500

Variant A:
  Users: 350
  Conversions: [random number between 0 and 350, likely around 17-18]
  Conversion Rate: [e.g., 0.051]

Variant B:
  Users: 150
  Conversions: [random number between 0 and 150, likely around 12]
  Conversion Rate: [e.g., 0.085]

Winner: Variant B

Explanation: 500 users are split with 70% to A (350 users) and 30% to B (150 users). Variant A has a 5% conversion probability, and Variant B has an 8% conversion probability. Even with fewer users, Variant B's higher conversion rate leads to it being the winner.

Example 3: Edge Case - Zero Users

Input: total_users = 0 split_ratio = {'A': 0.5, 'B': 0.5} conversion_probabilities = {'A': 0.1, 'B': 0.12}

Output:

A/B Test Results:
--------------------
Total Users: 0

Variant A:
  Users: 0
  Conversions: 0
  Conversion Rate: 0.0

Variant B:
  Users: 0
  Conversions: 0
  Conversion Rate: 0.0

Winner: No winner (or draw, as no data)

Explanation: When there are no users, no conversions can occur, and the conversion rates are 0. There is no meaningful winner.

Constraints

• total_users will be a non-negative integer.
• split_ratio will be a dictionary with keys 'A' and 'B', where values are floats summing to 1.0 (e.g., {'A': 0.5, 'B': 0.5}).
• conversion_probabilities will be a dictionary with keys 'A' and 'B', where values are floats between 0.0 and 1.0 (inclusive).
• The simulation should run within reasonable time limits for up to 1,000,000 users.

Notes

• You will need to use Python's random module for simulating user assignment and conversions.
• Consider how to handle floating-point precision when calculating conversion rates.
• The "winner" is simply the variant with the highest observed conversion rate in the simulation. For a real-world A/B test, statistical significance testing (e.g., t-tests, chi-squared tests) would be necessary, but that is outside the scope of this challenge.
• Your solution can be implemented as a class (e.g., ABTester) or a set of functions.