Project Potato Peeler: Web Scraping, and New Stats?

[jacobcarson877 2,541]

I wanted to formally introduce my project and get some feedback, and maybe some questions that the community has that I might be able to answer. For those who haven’t been reading every single thing I’ve ever written, I am currently working on my portfolio so I can get a job in the scary field of programming things, and I’ve taken a real liking to Data Analysis and Visualization over the last couple of years. Unfortunately, all of my projects involve school resources, and having recently graduated, those aren’t going to be available to me forever. So my goal is to start building up some projects using things I care about, to support my degree on my resume.

 

So here is where you are involved. I want to showcase (and actually have) my talents with web scraping, text formatting and regression analysis. I have done some projects before using a couple seasons of data, manually copied and pasted, but I want to do some more work to make it far more simple, and to increase the scale of data.

 

I have some individual projects that I want to culminate into a relational database, tying users to players, teams, leagues and seasons. All of the stats the indices (that we have of course) along with a few of my own creation (more on that later), in some form of searchable and exportable format.

 

I am currently practicing with player pages on the portal, pulling individual elements from those web-pages and working on creating the player tables. So far so good, just learning how the source of the portal is organized and how I can make logic that works for players and goalies before and after the hybrid era. This won’t end up in the final result, as far as I’m aware, but I want to get something working in an environment I’m familiar with.

 

The next one to tackle, and will definitely be harder, is the indices. I’ll start with scoring and roster pages, importing end of season, and end of playoff builds to make an impromptu build screenshot tool. I know that’s something that @emidas has made for the SBA and I think it’s really cool, even if I can’t perfectly replicate it.

 

The last item, and the interesting one to you folks is parsing the game commentary. I know many of us cry every time we go to the portal and see the toggle for giveaways and takeaways but find out the stats aren’t tracked. I also want to try to track some fun stats like zone exits and entries, and passing percentage. I’ll also grab some of the stats that only the index has, like own shots blocked and own shots missed, and hits taken. I also want to find out how many hits lead to turnovers.

 

If anyone has any other questions they want answered before the next phase of stuff (the actual regression analysis) let me know! What stats do you wish were tracked based on the game logs?

[Triller 1,668]

I'd love to see player stats against each team.  I track them manually at the moment and it sucks.  I'd also like to see combined points per teammate. I don't know what you'd specifically call it but it would show how many times I assisted on a specific teammates goals and how many times my player has assisted on a specific teammates goals.  Might shine a dim light on build compatibility and give GMs more specific line combos or trade and draft prospects?  Goals off rebounds would be fun to see or more broad, a tally of how your goals were scored.

[jacobcarson877 2,541]

1 hour ago, Triller said:

I'd love to see player stats against each team.  I track them manually at the moment and it sucks.  I'd also like to see combined points per teammate. I don't know what you'd specifically call it but it would show how many times I assisted on a specific teammates goals and how many times my player has assisted on a specific teammates goals.  Might shine a dim light on build compatibility and give GMs more specific line combos or trade and draft prospects?  Goals off rebounds would be fun to see or more broad, a tally of how your goals were scored.

All of these sound fun! I can imagine how they would be done, at least hypothetically!

 

[samx 1,173]

why did you name it project potato Peeler?

[jacobcarson877 2,541]

40 minutes ago, samx said:

why did you name it project potato Peeler?

i wanted an alliteration and peeling and scraping are like basically the same thing

[Gustav 6,430]

9 hours ago, jacobcarson877 said:

The next one to tackle, and will definitely be harder, is the indices. I’ll start with scoring and roster pages, importing end of season, and end of playoff builds to make an impromptu build screenshot tool. I know that’s something that @emidas has made for the SBA and I think it’s really cool, even if I can’t perfectly replicate it.

 

We have this on Google Sheets in a very limited fashion and I actually picked up where it left off at one point in March 2020 (could you imagine why?). I stopped doing it when I hit (or at least got very close to) a point where the update scale changed or something like that happened where I'd have to make changes to the back end of the sheet to keep it going. 

 

This was a giant pain because I had to write every player's name individually and copy a lot of links to a lot of places. Also no one really seemed to care. Seems like over 3 years later I still haven't learned that would happen. If it can be done easier, that would be really cool though!