+-+-+ +-+ +-+-+-+-+
|G|O| |4| |H|I|V|E|
+-+-+ +-+ +-+-+-+-+

 --- A GOPHER-LIKE INTERFACE FOR HIVE BLOCKCHAIN ---

Example of Steemit Analysis by Using Python #Tutorial-1

BY: @onderakcaalan | CREATED: March 20, 2018, 10:17 p.m. | VOTES: 75 | PAYOUT: $47.05 | [ VOTE ]

What Will I Learn?

This tutorial covers the topics on obtaining information from the internet page by using Python.

Requirements

For this tutorial, you need Python 3.6.4 (actually you do not need newest version but always updating the Python is useful for you.) and PyCharm program.

Difficulty

This tutorial has an indermadiate level.

Tutorial Contents

We want to build a program to make an analysis for Steemit members such as personalized tags, blog headings, wallet etc. Today, we will show how you can take information from a website by using Python.

First of we need to setup Python for Windows to use it. For this issue, below pictures are easy to understand and make setup:

[IMAGE: https://res.cloudinary.com/hpiynhbhq/image/upload/v1521577690/pmomiayqy0cv6t56qz4h.png]

Then we need to setup PyCharm program to work Python and you can download it like this:

[IMAGE: https://res.cloudinary.com/hpiynhbhq/image/upload/v1521577778/dstpqyetiwvjecy4zbzi.png]

After setup Python and PyCharm program, you need to open PyCharm program and the design will be like this and you can create files by right clicking the folder which is named untitled1 for us. Then you need to choose new and then Python file. After giving a name for file you are ready.

[IMAGE: https://res.cloudinary.com/hpiynhbhq/image/upload/v1521578113/na2nepfnddgucmufsa2i.png]

Now, you are ready to write your code in Python. Before starting to write the codes we need some packages which is written programmers and we can use these packages for free. We like Python due to its open source. Also you can create your packages and you can advertise it. Anyway, let's learn how to download packages. You need to choose File>Settings then there will be a pop-up menu which is related to general settings and also Project Interpreter menu. You can see the dowloaded packages and also you can add all packages here.

[IMAGE: https://res.cloudinary.com/hpiynhbhq/image/upload/v1521578503/j5w7dp39i4hwhwzsehqc.png]

For our tutorial we need to dowload beautifulsoup4 and requests packages.

[IMAGE: https://res.cloudinary.com/hpiynhbhq/image/upload/v1521578663/ce2htvfjpyowyymtgwdu.png]

We want to skip some basic learnings such as math operations, strings, variables, print etc. Also we think that with a written program you can learn a program much more easier. To achieve our goal for this tutorial, we will use two important operations which are for loop and if statement. We will explain these operations also into the code.

First of all, we need import the packages which we need to use in our program.

[IMAGE: https://res.cloudinary.com/hpiynhbhq/image/upload/v1521579074/j3xlgosdjvwocxkkk8ls.png]

Now, we need to find a website to get information from it. We used our Steemit homepage which is "https://steemit.com/@onderakcaalan" Lets see, what kind of info we can take from that page. To do that you can right click the page and choose Inspect, there wil be,

[IMAGE: https://res.cloudinary.com/hpiynhbhq/image/upload/v1521579490/odbe21pg058ljf3yencl.png]

in the Sources tag. These are the info which you can play:D. You can download it for yourself or make an analysis from the data or what ever you want. To get these info we will use requests command. Also we want to take html part of the wabpage which contains good sources by using BeautifulSoup package command. Here is the example:

[IMAGE: https://res.cloudinary.com/hpiynhbhq/image/upload/v1521581372/siiqdsq4o6k28r1snb2v.png]

If you want to see the results you can just write print command:

[IMAGE: https://res.cloudinary.com/hpiynhbhq/image/upload/v1521581713/xgvyg3kr2mmqr0vzi0zk.png]

You can see, we have data which contains **"
Posted on Utopian.io - Rewarding Open Source Contributors

TAGS: [ #utopian-io ] [ #tutorials ] [ #python ] [ #steemit ] [ #busy ]

Replies

@scipio | March 21, 2018, 1:15 a.m. | Votes: 4 | [ VOTE ]

Thank you for the contribution. It has been approved.

Please consider this:
- as you can see, I'm the author of the Learn Python Series (as well as a Utopian mod & Advisor),
- therefore I think I'm in the position to properly judge the quality of this tutorial,
- I've had quite some doubts if I would approve or rejct this tutorial, and as you can see, in the end I decided to approve it, but barely,
- there are quite some things missing and other aspects that could be improved, such as:

  • your English spelling & grammar could use a few improvements;
  • the PyCharm Professional IDE is not open source (minor issue). You could have easily discussed in a few steps PyCharm Community Edition (which is),
  • you forgot to explain to the user they first need to (pip) install both requests and beautifulsoup4 before they are able to import it: you are (probably) not using a virtual environment, meaning you've installed them at an earlier date. But users who have not installed them, or are using a new virtual environment per project, would get errors and can therefore not complete the tutorial;
  • you haven't explained what requests is (= a simple HTTP library for Python), or what beautifulsoup4 is (an HTML parsing library),
  • your tutorial title is mis-leading somewhat: you are explaining - in essence - how to use requests to fetch a web page, and beautifulsoup4 to parse the html using "jQuery-like" DOM-selectors, which can be used on any html web page, and thus has nothing to do with "analyzing steemit". Further, you're not analyzing anything, only parsing some DOM-elements and assigning them to variables;
  • you are not properly explaining any of the bs4 methods (why use find_all('a') for example? I know, but others don't. Are there other methods they can use?)
  • you are only printing the data you've fetched from the webpage. The minimum you could have done is wrap a more modular function around some bs4 code and return useful data from it.

Anyway, you are hereby strongly encouraged to seriously upgrade the quality of your next Python tutorials.

You can contact us on Discord.
[utopian-moderator]

@onderakcaalan | March 21, 2018, 1:31 a.m. | Votes: 0 | [ VOTE ]

Thanks for your encouragement. We will try to explain the issues according to yout comments in the next tutorial. Also we are very happy and appreciate to see such a serious comments from you. We are newbie among you but we will improve ourselves and our posts, be sure.

@utopian.tip | March 21, 2018, 6:56 p.m. | Votes: 1 | [ VOTE ]

Hey @scipio, I just gave you a tip for your hard work on moderation. Upvote this comment to support the utopian moderators and increase your future rewards!

@utopian-io | March 21, 2018, 5:10 a.m. | Votes: 1 | [ VOTE ]

Hey @onderakcaalan I am @utopian-io. I have just upvoted you!

Achievements

  • You have less than 500 followers. Just gave you a gift to help you succeed!
  • This is your first accepted contribution here in Utopian. Welcome!

Community-Driven Witness!

I am the first and only Steem Community-Driven Witness. Participate on Discord. Lets GROW TOGETHER!
- Vote for my Witness With SteemConnect
- Proxy vote to Utopian Witness with SteemConnect
- Or vote/proxy on Steemit Witnesses

[IMAGE: https://steemitimages.com/DQmYPUuQRptAqNBCQRwQjKWAqWU3zJkL3RXVUtEKVury8up/mooncryption-s-utopian-io-witness-gif.gif]

Up-vote this comment to grow my power and help Open Source contributions like this one. Want to chat? Join me on Discord https://discord.gg/Pc8HG9x

@onderakcaalan | March 21, 2018, 7:02 a.m. | Votes: 0 | [ VOTE ]

Thanks to @utopian-io community...

@bitgeek | March 21, 2018, 7:36 a.m. | Votes: 1 | [ VOTE ]

Congratulations @onderakcaalan, this post is the forth most rewarded post (based on pending payouts) in the last 12 hours written by a Newbie account holder (accounts that hold between 0.01 and 0.1 Mega Vests). The total number of posts by newbie account holders during this period was 3218 and the total pending payments to posts in this category was $1524.06. To see the full list of highest paid posts across all accounts categories, click here.

If you do not wish to receive these messages in future, please reply stop to this comment.

@onderakcaalan | March 21, 2018, 7:56 a.m. | Votes: 0 | [ VOTE ]

We like statistics so no problem...

@arcange | March 21, 2018, 4:38 p.m. | Votes: 1 | [ VOTE ]

Congratulations @onderakcaalan!
Your post was mentioned in the Steemit Hit Parade for newcomers in the following category:

  • Pending payout - Ranked 7 with $ 43,69

I also upvoted your post to increase its reward
If you like my work to promote newcomers and give them more visibility on Steemit, feel free to vote for my witness! You can do it here or use SteemConnect

@minnowsupport | March 21, 2018, 10:47 p.m. | Votes: 0 | [ VOTE ]

Congratulations! This post has been upvoted from the communal account, @minnowsupport, by onderakcaalan from the Minnow Support Project. It's a witness project run by aggroed, ausbitbank, teamsteem, theprophet0, someguy123, neoxian, followbtcnews, and netuoso. The goal is to help Steemit grow by supporting Minnows. Please find us at the Peace, Abundance, and Liberty Network (PALnet) Discord Channel. It's a completely public and open space to all members of the Steemit community who voluntarily choose to be there.
If you would like to delegate to the Minnow Support Project you can do so by clicking on the following links: 50SP, 100SP, 250SP, 500SP, 1000SP, 5000SP. Be sure to leave at least 50SP undelegated on your account.

@gokhan83kurt | March 22, 2018, 12:56 p.m. | Votes: 1 | [ VOTE ]

Adamın dediklerini dikkate al iyi gidiyorsun!

@arcange | March 22, 2018, 1 p.m. | Votes: 1 | [ VOTE ]

> First of all, I noticed that @arcange stopped the giving the analysis for the Steemit.

I'm not sure to understand what you are talking about. Can you elaborate on what I stopped to do?

@onderakcaalan | March 22, 2018, 1:11 p.m. | Votes: 0 | [ VOTE ]

Sorry, maybe we misunderstood but in this webpage which belongs to you, https://steemsql.com we think you stopped giving information about Steemit data without any payment. We want to say this issue. If we make a mistake about you, really sorry.

@arcange | March 22, 2018, 1:44 p.m. | Votes: 0 | [ VOTE ]
  1. The fact that the access to SteemSQL is no more free does not mean it has stopped to be available.
  2. What I am providing with SteemSQL is an access to a database and not an analysis service.

Your comment is misleading and confusing.

And now, some feedback about the content of your post.
Your approach might not be the best as you are extracting data from information displayed on a website.
Many information available in the blockchain are not available on the UI.
Then, have the owner of the website change its design, your code might fail.

@onderakcaalan | March 22, 2018, 2 p.m. | Votes: 0 | [ VOTE ]

You are right, i will delete about your part.
You are right also about code, but this is just an example for tutorial to explain some package, and other stuff.
Thanks for your valuable comments.

[ BACK TO TRENDING ] [ BACK TO MENU ]
CMD>