Project Euler / programming problems

Here is a problem I came up with yesterday:

Suppose we place 100 distinct lines in the plane such that there is no point where more than two lines intersect. How many possibilities are there for the total number of intersection points?

For example, with 3 lines it is possible to get:

  • 0 intersections (all three lines parallell)
  • 2 intersections (two lines parallell)
  • 3 intersections (no lines parallell)

but 1 intersection is not possible, since all three lines are not allowed to intersect in the same point. Thus there are 3 different possibilities for the total number of intersections.

9 Likes

I got an answer by brutal force. I don’t know if it’s correct.

Answer

3881

#include <iostream>
#include <cstring>
#include <unordered_set>

using namespace std;

int main(int argc, char* argv[]){

	if(argc < 2){
		return 1;
	}

	int n, m;
	n = atoi(argv[1]);
	if(n < 0){
		return 1;
	}

	int table[n];
	int count = 0;
	unordered_set<int> record;

	auto print = [&](int r){
		for(int i = 0; i < m; ++i){
			cout << table[i] << ",";
		}
		cout << " " << r;
		cout << endl;
	};

	auto calc = [&]() -> int {
		int result = 0;
		for(int i = 0; i < m - 1; ++i){
			for(int j = i + 1; j < m; ++j){
				result += table[i] * table[j];
			}
		}
		return result;
	};

	for(m = 1; m <= n; ++m){
		for(int i = 0; i < m - 1; ++i){
			table[i] = 1;
		}
		table[m - 1] = n - m + 1;
		int r = calc();
		//print(r);
		++count;
		record.insert(r);

		bool flag = true;
		while(flag){
			int sum = table[m - 1];
			flag = false;
			for(int i = m - 2; i >= 0; --i){
				sum += table[i];
				int t = sum - (table[i] + 1) * (m - i);
				if(table[i] < table[i + 1] && t >= 0){
					int a = table[i] + 1;
					for(int j = i; j < m; ++j){
						table[j] = a;
					}
					table[m - 1] += t;
					int r = calc();
					//print(r);
					++count;
					record.insert(r);
					flag = true;
					break;
				}
			}
		}
	}

	cout << "count: " << count << endl;
	cout << "unique: " << record.size() << endl;

	return 0;
}
$ g++ -O3 test.cc -o test
$ time ./test 100
count: 190569292
unique: 3881

real	0m21.434s
user 0m21.426s
sys	0m0.008s

2 Likes
Your answer is correct!

I can’t follow all the code, but I believe it is exponential time complexity since it seems to iterate over all partitions of 100?

It can be done in polynomial time - I have an O(N^4) solution, but a friend of mine thinks it’s possible to go sublinear, so I’m excited to see how far this can be pushed by people smarter than me :smile:

1 Like

Oh, I think i got your solution.

Summary

lagrida_latex_editor
and dynamic programming

3 Likes

Yep!

Making a few calculations in my head, this problem is equivalent to finding the number of different values for the sum of squares of natural numbers that add up to 100.

4 Likes

Now determine the number of distinct intersection patterns :smiley:
(that is - if I remember correctly - lines in the plane modulo an equivalence relation where we look for a mapping between the lines such that the intersections occur in the same order from the perspective of each line)

Spoiler

Some years ago a professor of mine said they were calculuting this for 21 lines, with a computation time of over a month.

1 Like

Here is a DP approach. It’s not very efficient and has a lot of redundancy. It takes a noticeable fraction of a second to get the result for 100.

Project_antonToby.py
from functools import cache

@cache
def sums_squares(n):
    if n==0:
        return {0}
    else:
        result=set()
        for k in range(1,n+1):
            k2=k*k
            result.update(k2+s for s in sums_squares(n-k))
        return result

def f(n):
    return len(sums_squares(n))

Also found the OEIS sequence. Unfortunately, “No generating function is known”.

2 Likes

That’s a nicer DP solution than the one I had (mine took about 10 seconds to get the result for 100).

Ugly code
cache = { (0,0): True }

# returns true if n can be "partition-squared" into k
def f(n, k):
    if n*n < k:
        return False
    try:
        return cache[n, k]
    except:
        pass
    for i in range(1, n+1):
        if f(n-i, k-i*i):
            cache[n, k] = True
            return True
    cache[n, k] = False
    return False


n = 100
count = 0
for k in range(n*n+1):
    if f(n, k):
        count += 1

print(count)

I can’t tell if yours is better than O(N^4) or still O(N^4) but with a much lower constant - I guess assuming the length of sums_squares(n) grows quadratically it’s still O(N^4)?

Does anyone know how to go lower? :smiley:

3 Likes

I think the biggest python sin of your code is using try and except while expecting to hit an exception somewhat often. Have you compared if an if would do better?

Like this?

if (n, k) in cache:
   return cache[n, k]

Yeah that’s nicer :slight_smile: Actually I’ve written it like this before, but then I saw the try-except variant in some example I looked up yesterday and thought “hey, maybe that’s better for some reason?”. Something about first checking whether a value is in the dict and then looking it up feels redundant to me, but I guess the exception is at least as costly. And it makes sense that expecting to get an exception is not best practice :smile:

Anyways, that change on its own didn’t make my code much faster - I think the bigger performance difference is that I generate a full table of True/False values while you only store and iterate over the “True” values so to speak?

I should remember the convenient functools.cache for next time - I know about it but keep forgetting it exists because I don’t do stuff like this very often :smile:

1 Like

It’s situational. try…except is fine if you expect not to hit an exception most of the time (for some fuzzy definition of “most”).
In your case I think you may want to try something like this:

x=cache.get((n,k))
if x is None:
    ...
else:
    ...

but you’re right, you’re probably storing a lot more values than needed.

Glass houses. I once spent like a whole month creating and fine tuning a very inefficient module that had the same exact purpose as the functools.cache decorator. I didn’t even know it existed at the time :grimacing:

EDIT:

here it is!

Take a moment to appreciate the irony that I actually imported something else from functools :person_facepalming:.

#dynamic_programming.py

from functools import wraps
from inspect import signature, Signature, Parameter
from collections import namedtuple


class Dynamic_Memory():

    __nullref = type('CustomNullPointer',(),
                    {'__str__' : lambda self: '<NULL>',
                     '__repr__': lambda self: '<NULL>',
                     '__bool__': lambda self: False})()
    # Alternatively:
    # __nullref = None
    # This has the disadvantage of not recognizing 'None' as the result of calculation
    
    
    __ParameterInfo = namedtuple("ParameterAsociationsWithCache", ['parameter', 'container_type', 'offset'])
    __MemoryStats = namedtuple("CacheInfoStats", ['containers', 'null_pointers', 'stored_values', 'depth'])
    
    @classmethod
    def null_pointer(cls):
        return cls.__nullref
        
    ############################
    ##     basic methods      ##
    ############################
    
    def __init__(self, function, *, container_type = list, offset = None, init_memory = None, custodian = None):
        self.__function = function
        self.__calls = 0
        self.__custodian = custodian
        self.__sig = signature(function)
        
        params = self.__sig.parameters
        self.__container_dimension = len(params)
        
        try:
            assert self.__container_dimension > 0
            for param in params.values():
                assert Parameter.VAR_POSITIONAL is not param.kind is not Parameter.VAR_KEYWORD
        except AssertionError:
            raise ValueError("Dynamic_Memory class requires {}() to have a fixed non-zero number of arguments.".format(function.__qualname__)) from None
    
        try:
            assert self.__container_dimension == len(container_type)
        except TypeError:
            container_type = (container_type,)*self.__container_dimension
        except AssertionError:
            raise ValueError("Dynamic_Memory class requires container_type argument to be either a container class or a sequence of container classes whose len matches the number of arguments of {}().".format(function.__qualname__)) from None            
        self.__container = init_memory if init_memory else container_type[0]()
    
        try:
            assert self.__container_dimension == len(offset)
        except TypeError:
            offset = (offset,)*self.__container_dimension
        except AssertionError:
            raise ValueError("Dynamic_Memory class requires offset argument to be either a numeric type or a sequence of numeric types whose len matches the number of arguments of {}().".format(function.__qualname__)) from None
        
        self.__zipped_info = [self.__ParameterInfo(p,c,o) for p,c,o in zip(params, container_type, offset)]

    
    def __del__(self):
		# Probably redundant
        del self.__container
    
    
    ############################
    ##   emulation methods    ##
    ############################
    
    def __call__ (*args, **kwargs):
        self, args = args[0], args[1:]
        self.__calls += 1
        
        result, this_container, pos = self.__access_memory(args, kwargs)
        
        if result is self.__nullref:
            result = this_container[pos] = self.__function (*args, **kwargs)
        
        return result
    
    
    def __getitem__(self, key):
        return self.__access_memory_direct(*self.__subscript_to_arguments(key))

    
    def __setitem__ (self, key, value):
        args, kwargs = self.__subscript_to_arguments(key)
        
        in_cache, container, pos = self.__access_memory(args,kwargs)
        if in_cache is not self.__nullref:
            raise ValueError("{}() has already calculated the value at {} {}.".format(self.__function.__qualname__,args, kwargs))
        else:
            container[pos]= value
    
    def __contains__(self, key):
        return self.__access_memory_direct(*self.__subscript_to_arguments(key), return_bool = True)
        
            
    def __str__(self):
        name = str(self.__function)
        if len(name)>0 and name[0]=='<' and name[-1]=='>':
            name = name[1:-1]
        return "<{}, with dynamic memory>".format(name)
            
    def __repr__(self):
        name = repr(self.__function)
        if len(name)>0 and name[0]=='<' and name[-1]=='>':
            name = name[1:-1]
        return "<{}, with dynamic memory>".format(name)
        
    
    def __getattr__(self, name):
        return self.__function.__getattribute__(name)
        
    def __dir__(self):
        prefix = "_Dynamic_Memory"
        plen = len (prefix)
        return [name for name in set(super().__dir__()).union(dir(self.__function)) if prefix != name[:plen]]
        
    ############################
    ##     public methods     ##
    ############################
    
    def __reset_cache (self):
        self.__container = self.__zipped_info[0].container_type()
    cache = property(lambda self: self.__container, None, __reset_cache, "Dynamically stored memory.")
    
    function = property (lambda self: self.__function, None, None, "Original function")
    dimensions = property (lambda self: self.__container_dimension, None, None, "Dimensions of cache memory. Same as the number of arguments of the function")
    storage_info = property (lambda self: self.__zipped_info.copy(), None, None, "Information about how parameters are stored in cache.")
    call_count = property (lambda self: self.__calls, None, None, "Number of times the function has been invoked (including already calculated values).")
        
    def full_memory_stats(self):
        containers = 0
        null_pointers = 0
        data = 0
        depth = 1
        
        def memory_inspector (container, argsleft, null, level):
            nonlocal containers, null_pointers, data, depth

            if depth < level:
                depth = level
            containers += 1
            try:
                iterable = container.values()
            except AttributeError:
                iterable = container
            
            if argsleft:
                for value in iterable:
                    if value is null:
                        null_pointers += 1
                    else:
                        memory_inspector (value, argsleft-1, null, level + 1)
            else:
                for value in iterable:
                    if value is null:
                        null_pointers += 1
                    else:
                        data +=1
        
        memory_inspector(self.__container, self.__container_dimension - 1, self.__nullref, 1)
        
        return self.__MemoryStats(containers, null_pointers, data, depth)
        

    
    ############################
    ##    utility methods     ##
    ############################
        
    def __access_memory(self, args, kwargs):
        custodian = self.__custodian
        boundargs = self.__sig.bind(*args,**kwargs)
        if custodian:
            if not custodian(*args, **kwargs):
                raise ValueError("{}() custodian function does not allow the following arguments to be stored in cache: {} {}".format(self.__function.__qualname__,args,kwargs))
        boundargs.apply_defaults()
        boundargs = boundargs.arguments
        
        pos = 0
        this_container = None
        this_reference = self.__container
        nullref = self.__nullref
        for name, ctype, ofs in self.__zipped_info:
            if this_reference is nullref:
                this_container[pos] = this_reference = ctype()
            
            if ofs:
                pos = boundargs[name] - ofs
            else:
                pos = boundargs[name]
            
            this_container = this_reference
            try:
                this_reference = this_container[pos]
            except IndexError:
                try:
                    this_container.extend([nullref]*(pos-len(this_container)+1))
                    this_reference = nullref
                except Exception as e:
                    raise e from None
            except KeyError:
                try:
                    this_reference = this_container[pos] = nullref
                except Exception as e:
                    raise e from None
        
        return this_reference, this_container, pos
    
    
    def __access_memory_direct(self, args, kwargs, return_bool = False):
        # Does not create an entry
        boundargs = self.__sig.bind(*args,**kwargs)
        boundargs.apply_defaults()
        boundargs = boundargs.arguments
        
        pos = 0
        this_reference = self.__container
        nullref = self.__nullref
        for name, ctype, ofs in self.__zipped_info:
            if this_reference is nullref:
                if return_bool:
                    return False
                else:
                    raise KeyError("{}() has not yet calculated a value at {} {}.".format(self.__function.__qualname__,args, kwargs)) from None
            
            if ofs:
                pos = boundargs[name] - ofs
            else:
                pos = boundargs[name]
            
            try:
                this_reference = this_reference[pos]
            except (IndexError, KeyError):
                if return_bool:
                    return False
                else:
                    raise KeyError("{}() has not yet calculated a value at {} {}.".format(self.__function.__qualname__,args, kwargs)) from None
        
        if this_reference is nullref:
            if return_bool:
                return False
            else:
                raise KeyError("{}() has not yet calculated a value at {} {}.".format(self.__function.__qualname__,args, kwargs)) from None
        elif return_bool:
            return True
        else:
            return this_reference
        
        
    def __subscript_to_arguments(self, key):
        if type(key) is not tuple:
            key=(key,)
        
        dimensions = self.__container_dimension
        try:
            assert dimensions == len(key)
        except AssertionError:
            raise TypeError("{}() requires exactly {} arguments to access its cache memory. Given: {}.".format(self.__function.__qualname__, self.__container_dimension,len(key))) from None
        
        lastarg = dimensions
        kwargs = {}
        while lastarg > 0:
            arg = key[lastarg-1]
            if type(arg) is slice:
                k = arg.start
                v = arg.stop
                if k in kwargs:
                    raise TypeError("{}() got multiple values for keyword argument {}".format(self.__function.__qualname__, k))
                else:
                    kwargs[k]=v
                lastarg -= 1
            else:
                break
            
        return key[:lastarg], kwargs
    ###########################

 
def dynamic_memory(**kwargs):
	# Should have called it "memoization". Too late to back up now.
    return lambda function: wraps(function)(Dynamic_Memory(function,**kwargs))
1 Like

Ok, last improvements, and I’ll leave it at that:

  1. I take advantage of the fact that if there are at least two numbers that sum to n, one of them is smaller than half of n.
  2. All sums of squares have the same parity.
  3. The possible sums of squares create very dense intervals, so, instead of storing the numbers individually, I store their ranges (sorted in order)

So, there is an additional difficulty of creating a merging function, but it is not too outlandish

Project_antonToby.py
from functools import cache

def merge_iter(iter1,iter2):
    finished=False
    try:
        r1 = next(iter1)
    except StopIteration:
        yield from iter2
        finished=True
    try:
        r2 = next(iter2)
    except StopIteration:
        yield r1
        yield from iter1
        finished=True
    while not finished:
        if r2.start < r1.start:
            r1,r2=r2,r1
            iter1,iter2=iter2,iter1
        yield r1
        try:
            r1=next(iter1)
        except StopIteration:
            yield r2
            yield from iter2
            finished=True

def union(iter1,iter2):
    iter1=iter(iter1)
    iter2=iter(iter2)
    result=[]
    for r in merge_iter(iter1,iter2):
        if result:
            r0=result[-1]
            if r.start <= r0.stop:
                r0=range(r0.start, max(r0.stop, r.stop),2)
                result[-1]=r0
            else:
                result.append(r)
        else:
            result.append(r)
    return result

@cache
def sums_squares(n):
    if n==0:
        return [range(0,2,2)]
    else:
        n2=n*n
        result=[range(n2,n2+2,2)]
        for k in range(n//2,0,-1):
            k2=k*k
            result=union(
                result,
                (range(k2+r.start,k2+r.stop,2) for r in sums_squares(n-k)))
        return result

def f(n):
    return sum(len(r) for r in sums_squares(n))


2 Likes

Well, it’s been almost a year, but I finally made it to level 15 :blush:. I guess I’ll keep crawling towards the next goal, though this will probably take the back seat for a while.

1 Like

Here is a little problem I played around with today:

Let’s say we roll two regular (6-sided) dice, and compute the product of the two numbers that come up. What is the most likely product to show up? In other words, if you had to bet on a specific product showing up, which number should you bet on?

It’s a tie between 6 and 12, since both of these can be written in 4 different ways:
image
image

However, if we do the same thing with five dice, it turns out that there is a unique “most frequent product”: 360 can be written in 300 different ways as a product of five numbers between 1 and 6, and there is no other number that beats (or ties) this.

Next, let’s say we throw 361 dice. Is there a unique most frequent product? If so, what is this product?

I believe this is probably possible for a clever person to solve using just pen and paper. My own solution so far is part math, part code, and part conjecture, so I’m not 100% confident I have the correct answer :slight_smile:

6 Likes

Much easier with a 3 sided die :sweat_smile:

If we completely ignore the fact that 4 and 6 can be factored into 2s and 3s, it looks like it’s got to be somewhere in the ballpark of 8171. That is, 6!N/6

I ran a simulation, and plotted counts. It does look like a bell centered around that value if x is log scale:

my money's on...

6!60 ~= 2.75e171

You can see a little dot that hops up above the peak in the graph. Just a guess though.

2 Likes

I think this kind of reasoning will get close to something like the “average” product, but not necessarily the most frequent product.

I’m having trouble reconciling your simulation with the answer I got though, so now I’m starting to doubt myself… could you run the same simulation for 5 dice and see how that plot looks?

Edit: Also, did you compute the products exactly or with limited precision?

1 Like

Still in that ballpark of 6!5/6=240.5, but as you noted that reasoning doesn’t get us all the way there. The simulation shows 360 as a favorite. (it’s funny to run this random simulation, when the proper enumeration would be faster :joy:)

I computed products exactly. Python BigInts are probably why I couldn’t run a very long simulation - just 1M samples took a while. I did convert the x values to float before feeding it to matplotlib, but I don’t think that would cause issues.

1 Like

Welp, I’m still confused then :sweat_smile:

I got an answer with 159 digits, but if you are able to run a simulation long enough that some exact products are showing up >100 times, shouldn’t my “most frequent product” be at least getting close to 100 times, if it really is the theoretical answer?

(so maybe I’ve just made a mistake somewhere :slightly_smiling_face:)

1 Like