Mean Variance Optimization - Stuck with Jupyter

I am working on the Mean Variance Optimization exercise/project in the course Analyze Financial Data with Python.

I am doing this exercise/project in Jupyter instead of in Codecademy’s output terminal.

I am running into a big error in Step #7 where I am attempting to use the optimal_portfolio function. This error is way over my head because it is coming from one of the packages this function uses, which has some pretty deep math in it that I am not familiar with (I could learn it, but it would take me several weeks!). I am not sure how to share things from Jupyter here, so I will copy and past the error below.

Can anyone help me with this? It seems to me that the function and all the packages it uses should run just fine without such deep errors. So I am thinking that some little thing is off somewhere in my work. Or, perhaps this was not set up properly in Jupyter by Codecademy because a few other things were missing also (easy stuff I could figure out). I am completely stumped!

Here is the code and the error message:

7. Calculate the set of portfolios on the EF

Here I am applying the function, which is the same as what was used in the previous sections of the course

weights, returns, risks = optimal_portfolio(returns_quarterly)

Here is the massive error message:

 pcost       dcost       gap    pres   dres

0: nan nan nan nan nan


ValueError Traceback (most recent call last)
Cell In[11], line 2
1 # 7. Calculate the set of portfolios on the EF
----> 2 weights, returns, risks = optimal_portfolio(returns_quarterly)

Cell In[10], line 22, in optimal_portfolio(returns)
19 b = opt.matrix(1.0)
21 # Calculate efficient frontier weights using quadratic programming
—> 22 portfolios = [solvers.qp(mu*S, -pbar, G, h, A, b)[‘x’] for mu in mus]
23 ## CALCULATE RISKS AND RETURNS FOR FRONTIER
24 returns = [blas.dot(pbar, x) for x in portfolios]

Cell In[10], line 22, in (.0)
19 b = opt.matrix(1.0)
21 # Calculate efficient frontier weights using quadratic programming
—> 22 portfolios = [solvers.qp(mu*S, -pbar, G, h, A, b)[‘x’] for mu in mus]
23 ## CALCULATE RISKS AND RETURNS FOR FRONTIER
24 returns = [blas.dot(pbar, x) for x in portfolios]

File ~/miniconda3/lib/python3.11/site-packages/cvxopt/coneprog.py:4485, in qp(P, q, G, h, A, b, solver, kktsolver, initvals, **kwargs)
4475 pinfres, dinfres = None, None
4477 return {‘status’: status, ‘x’: x, ‘s’: s, ‘y’: y, ‘z’: z,
4478 ‘primal objective’: pcost, ‘dual objective’: dcost,
4479 ‘gap’: gap, ‘relative gap’: relgap,
(…)
4482 ‘residual as primal infeasibility certificate’: pinfres,
4483 ‘residual as dual infeasibility certificate’: dinfres}
→ 4485 return coneqp(P, q, G, h, None, A, b, initvals, kktsolver = kktsolver, options = options)

File ~/miniconda3/lib/python3.11/site-packages/cvxopt/coneprog.py:2243, in coneqp(P, q, G, h, dims, A, b, initvals, kktsolver, xnewcopy, xdot, xaxpy, xscal, ynewcopy, ydot, yaxpy, yscal, **kwargs)
2229 return { ‘x’: x, ‘y’: y, ‘s’: s, ‘z’: z, ‘status’: status,
2230 ‘gap’: gap, ‘relative gap’: relgap,
2231 ‘primal objective’: pcost, ‘dual objective’: dcost,
2232 ‘primal infeasibility’: pres,
2233 ‘dual infeasibility’: dres, ‘primal slack’: -ts,
2234 ‘dual slack’: -tz , ‘iterations’: iters }
2237 # Compute initial scaling W and scaled iterates:
2238 #
2239 # W * z = W^{-T} * s = lambda.
2240 #
2241 # lmbdasq = lambda o lambda.
→ 2243 if iters == 0: W = misc.compute_scaling(s, z, lmbda, dims)
2244 misc.ssqr(lmbdasq, lmbda, dims)
2247 # f3(x, y, z) solves
2248 #
2249 # [ P A’ G’ ] [ ux ] [ bx ]
(…)
2253 # On entry, x, y, z containg bx, by, bz.
2254 # On exit, they contain ux, uy, uz.

File ~/miniconda3/lib/python3.11/site-packages/cvxopt/misc.py:285, in compute_scaling(s, z, lmbda, dims, mnl)
275 # For the ‘l’ block:
276 #
277 # W[‘d’] = sqrt( sk ./ zk )
(…)
281 # where sk and zk are the first dims[‘l’] entries of s and z.
282 # lambda_k is stored in the first dims[‘l’] positions of lmbda.
284 m = dims[‘l’]
→ 285 W[‘d’] = base.sqrt( base.div( s[mnl:mnl+m], z[mnl:mnl+m] ))
286 W[‘di’] = W[‘d’]**-1
287 lmbda[mnl:mnl+m] = base.sqrt( base.mul( s[mnl:mnl+m], z[mnl:mnl+m] ) )

ValueError: domain error

You must select a tag to post in this category. Please find the tag relating to the section of the course you are on E.g. loops, learn-compatibility

When you ask a question, don’t forget to include a link to the exercise or project you’re dealing with!

If you want to have the best chances of getting a useful answer quickly, make sure you follow our guidelines about how to ask a good question. That way you’ll be helping everyone – helping people to answer your question and helping others who are stuck to find the question and answer! :slight_smile:

In case anyone else runs into the same problem, I solved this issue.

First, I should point out that the Jupyter Notebook created for this exercise by Codecademy does not load panda. So, you need to install pandas into the environment. You need to do the same for matplotlib.

Now, doing that brings in the latest version of pandas. And, herein is the problem. Codecademy is written for an older version of pandas.

The problem is that the code for the function optimal_portfolio(returns) that Codecademy provides has a method that is no longer used in the latest version of pandas.

This line has the method as_matrix which no longer exists pandas (they call it deprecated):

returns = np.transpose(returns.as_matrix())

So, you need to replace that line with the following line of code:

returns = np.transpose(returns.to_numpy())


I hope this helps some people out there that are trying to complete this course.

I wish Codecademy had better quality control. I suppose there was some educational benefit in tracking down this problem and solving it. (To be 100% truthful, I relied heavily on ChatGPT4 to figure it out). But it seems to me that this was a problem that was way out of the picture in terms of what we are learning in this course. It just used up an entire day for to figure it out, which would have been better spent on the Python topics of this course.

This sounds frustrating.

What is the link to the lesson?

So, the version installed on the LE was pre 2.1.1? Did the code work properly in the lesson?
If not, then it’s a bug and you can report it.

Yep, you need to have Pandas, Matplotlib, etc. installed on your machine. Sometimes, it’s better to do the lessons within the learning environment b/c to pass them they’ve installed packages and tests/code in the background to see if one’s answers are correct. But, I commend you for going off platform!

Rather than rely on CGPT (who knows where the training data came from for the LLM and the veracity of the info?), it’s always good practice to check the docs:

https://pandas.pydata.org/docs/user_guide/index.html#user-guide

From there, you can also see what methods are deprecated.

Also, fwiw, if you’re going to share code from a jupyter notebook, you can always upload your notebook to a GitHub repo and then share the link here. It’s a little better than trying to decipher a wall of text. :slight_smile: