Skip to content

  • Projects
  • Groups
  • Snippets
  • Help
  • This project
    • Loading...
  • Sign in / Register
V
VenJob
  • Overview
    • Overview
    • Details
    • Activity
    • Cycle Analytics
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Charts
  • Issues 0
    • Issues 0
    • List
    • Board
    • Labels
    • Milestones
  • Merge Requests 1
    • Merge Requests 1
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Charts
  • Wiki
    • Wiki
  • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Charts
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • Đường Sỹ Hoàng
  • VenJob
  • Merge Requests
  • !2

Merged
Opened Dec 05, 2019 by Đường Sỹ Hoàng@hoangds 
  • Report abuse
Report abuse

Jobs Crawler

Edited Dec 06, 2019 by Đường Sỹ Hoàng
×

Check out, review, and merge locally

Step 1. Fetch and check out the branch for this merge request

git fetch origin
git checkout -b feature/crawler origin/feature/crawler

Step 2. Review the changes locally

Step 3. Merge the branch and fix any conflicts that come up

git checkout master
git merge --no-ff feature/crawler

Step 4. Push the result of the merge to GitLab

git push origin master

Note that pushing to GitLab requires write access to this repository.

Tip: You can also checkout merge requests locally by following these guidelines.

  • Discussion 12
  • Commits 5
  • Pipelines 5
  • Changes 9
{{ resolvedDiscussionCount }}/{{ discussionCount }} {{ resolvedCountText }} resolved
  • Son Do Hong
    @sondh started a discussion on an old version of the diff Dec 05, 2019
    Last updated by Đường Sỹ Hoàng Dec 06, 2019
    lib/tasks/job.rake 0 → 100644
    57 puts job_page.css("div.LeftJobCB").css("div.MarBot20").text
    58 # puts job_page.css("div.LeftJobCB").css("div.MarBot20").css("div.content_fck").css("p").text
    59
    60 # detail company
    61 puts job_page.css("div.box1Detail").css("p.TitleDetailNew").css("label").text
    62 end
    63 Job.create!(
    64 title: job_page.css("div.MyJobDetail").css("div.MyJobLeft").css("div.LeftJobCB").css("div.top-job").css("div.top-job-info").css("h1").text.to_s
    65 description: job_page.css("div.LeftJobCB").css("div.MarBot20").css("h4.TitleJobNew").text
    66 description: job_page.css("div.LeftJobCB").css("div.MarBot20").css("div.content_fck").css("p").text
    67 short_description: job_page.css("div.LeftJobCB").css('div.desc_company.content_fck').css('span#emp_collapse').text
    68 salary: job_page.css("div#showScroll.box2Detail").css("ul.DetailJobNew").css("li.bgLine2").css("p.fl_right").text
    69 requirement: job_page.css("div.LeftJobCB").css("div.MarBot20").css("h4.TitleJobNew").text
    70 requirement: job_page.css("div.LeftJobCB").css("div.MarBot20").css("div.content_fck").css("p").text
    71 position: job_page.css("div#showScroll.box2Detail").css("ul.DetailJobNew").css("li.bgLine1").css("p.fl_left").text.to_s)
    72 Job.save!
    • Son Do Hong @sondh commented Dec 05, 2019
      Master

      câu này để làm gì

      câu này để làm gì
    • Đường Sỹ Hoàng @hoangds

      changed this line in version 2 of the diff

      Dec 06, 2019

      changed this line in version 2 of the diff

      changed this line in [version 2 of the diff](https://gitlab.zigexn.vn/hoangds/VenJob/merge_requests/2/diffs?diff_id=4428&start_sha=f24e4f4bf352d2aae1c9a146774ce036efaf41f5#f7088fd37ab62eb9ccec3914041a48694bdadd6b_72_46)
      Toggle commit list
    Please register or sign in to reply
  • Son Do Hong
    @sondh started a discussion on an old version of the diff Dec 05, 2019
    Last updated by Đường Sỹ Hoàng Dec 06, 2019
    import.rb 0 → 100644
    1 require "csv"
    2 require "activerecord-import/base"
    3
    4 def import
    5 jobs=[]
    6 CSV.foreach("app/lib/Venjob.csv",headers: true) do |row|
    • Son Do Hong @sondh commented Dec 05, 2019
      Master

      không làm ở merge request này

      không làm ở merge request này
    • Đường Sỹ Hoàng @hoangds

      changed this line in version 2 of the diff

      Dec 06, 2019

      changed this line in version 2 of the diff

      changed this line in [version 2 of the diff](https://gitlab.zigexn.vn/hoangds/VenJob/merge_requests/2/diffs?diff_id=4428&start_sha=f24e4f4bf352d2aae1c9a146774ce036efaf41f5#effadd549dcb5566cd3a2552bf98326d60f3fe02_6_0)
      Toggle commit list
    Please register or sign in to reply
  • Son Do Hong
    @sondh started a discussion on an old version of the diff Dec 05, 2019
    Last updated by Đường Sỹ Hoàng Dec 06, 2019
    Gemfile
    30 30
    31 31 gem 'devise'
    32 32
    33 source "https://rubygems.org"
    • Son Do Hong @sondh commented Dec 05, 2019
      Master

      dòng này bị trùng

      dòng này bị trùng
    • Đường Sỹ Hoàng @hoangds

      changed this line in version 2 of the diff

      Dec 06, 2019

      changed this line in version 2 of the diff

      changed this line in [version 2 of the diff](https://gitlab.zigexn.vn/hoangds/VenJob/merge_requests/2/diffs?diff_id=4428&start_sha=f24e4f4bf352d2aae1c9a146774ce036efaf41f5#de3150c01c3a946a6168173c4116741379fe3579_33_29)
      Toggle commit list
    Please register or sign in to reply
  • Son Do Hong
    @sondh started a discussion on an old version of the diff Dec 05, 2019
    Last updated by Đường Sỹ Hoàng Dec 06, 2019
    Gemfile
    30 30
    31 31 gem 'devise'
    32 32
    33 source "https://rubygems.org"
    34
    35 gem "nokogiri"
    36
    37 gem "mechanize"
    38
    39 gem "pry"
    • Son Do Hong @sondh commented Dec 05, 2019
      Master

      pry để trong group development và test

      pry để trong group development và test
    • Đường Sỹ Hoàng @hoangds

      changed this line in version 2 of the diff

      Dec 06, 2019

      changed this line in version 2 of the diff

      changed this line in [version 2 of the diff](https://gitlab.zigexn.vn/hoangds/VenJob/merge_requests/2/diffs?diff_id=4428&start_sha=f24e4f4bf352d2aae1c9a146774ce036efaf41f5#de3150c01c3a946a6168173c4116741379fe3579_39_33)
      Toggle commit list
    Please register or sign in to reply
  • Son Do Hong
    @sondh started a discussion on an old version of the diff Dec 05, 2019
    Last updated by Đường Sỹ Hoàng Dec 06, 2019
    Gemfile
    30 30
    31 31 gem 'devise'
    32 32
    33 source "https://rubygems.org"
    34
    35 gem "nokogiri"
    36
    37 gem "mechanize"
    38
    39 gem "pry"
    40
    41 gem "rubysl-open-uri"
    42
    • Son Do Hong @sondh commented Dec 05, 2019
      Master

      giữa các khai báo gem viết liền

      giữa các khai báo gem viết liền
    • Đường Sỹ Hoàng @hoangds

      changed this line in version 2 of the diff

      Dec 06, 2019

      changed this line in version 2 of the diff

      changed this line in [version 2 of the diff](https://gitlab.zigexn.vn/hoangds/VenJob/merge_requests/2/diffs?diff_id=4428&start_sha=f24e4f4bf352d2aae1c9a146774ce036efaf41f5#de3150c01c3a946a6168173c4116741379fe3579_42_34)
      Toggle commit list
    Please register or sign in to reply
  • Son Do Hong
    @sondh started a discussion on an old version of the diff Dec 05, 2019
    Last updated by Đường Sỹ Hoàng Dec 06, 2019
    Gemfile
    30 30
    31 31 gem 'devise'
    32 32
    33 source "https://rubygems.org"
    34
    35 gem "nokogiri"
    36
    37 gem "mechanize"
    38
    39 gem "pry"
    40
    41 gem "rubysl-open-uri"
    42
    43 gem "activerecord-import"
    • Son Do Hong @sondh commented Dec 05, 2019
      Master

      xóa gem này

      xóa gem này
    • Đường Sỹ Hoàng @hoangds

      changed this line in version 2 of the diff

      Dec 06, 2019

      changed this line in version 2 of the diff

      changed this line in [version 2 of the diff](https://gitlab.zigexn.vn/hoangds/VenJob/merge_requests/2/diffs?diff_id=4428&start_sha=f24e4f4bf352d2aae1c9a146774ce036efaf41f5#de3150c01c3a946a6168173c4116741379fe3579_43_34)
      Toggle commit list
    Please register or sign in to reply
  • Son Do Hong
    @sondh started a discussion on an old version of the diff Dec 05, 2019
    Resolved by Son Do Hong Dec 06, 2019
    lib/tasks/job.rake 0 → 100644
    1 require "rubygems"
    2 require "open-uri"
    3 require "nokogiri"
    4 require "mechanize"
    5 require "pry"
    6
    7 namespace :Job do
    8 desc "Import data from crawler to database"
    9 task :import do
    10 puts "Starting import data to database"
    11 end
    12 task create: :environment do
    • Son Do Hong @sondh commented Dec 05, 2019
      Master

      đặt tên task mô tả nội dung của task ví dụ: crawl_data

      đặt tên task mô tả nội dung của task ví dụ: `crawl_data`
    • Đường Sỹ Hoàng @hoangds

      changed this line in version 5 of the diff

      Dec 06, 2019

      changed this line in version 5 of the diff

      changed this line in [version 5 of the diff](https://gitlab.zigexn.vn/hoangds/VenJob/merge_requests/2/diffs?diff_id=4433&start_sha=9711ba07dc2b6f34e1411f92b7ebd0c27a48fc2d#f7088fd37ab62eb9ccec3914041a48694bdadd6b_9_9)
      Toggle commit list
    Please register or sign in to reply
  • Son Do Hong
    @sondh started a discussion on an old version of the diff Dec 05, 2019
    Last updated by Đường Sỹ Hoàng Dec 06, 2019
    lib/tasks/job.rake 0 → 100644
    1 require "rubygems"
    2 require "open-uri"
    3 require "nokogiri"
    4 require "mechanize"
    5 require "pry"
    6
    7 namespace :Job do
    • Son Do Hong @sondh commented Dec 05, 2019
      Master

      namespace viết thường

      namespace viết thường
    • Đường Sỹ Hoàng @hoangds

      changed this line in version 2 of the diff

      Dec 06, 2019

      changed this line in version 2 of the diff

      changed this line in [version 2 of the diff](https://gitlab.zigexn.vn/hoangds/VenJob/merge_requests/2/diffs?diff_id=4428&start_sha=f24e4f4bf352d2aae1c9a146774ce036efaf41f5#f7088fd37ab62eb9ccec3914041a48694bdadd6b_7_7)
      Toggle commit list
    Please register or sign in to reply
  • Son Do Hong
    @sondh started a discussion on an old version of the diff Dec 05, 2019
    Last updated by Đường Sỹ Hoàng Dec 06, 2019
    lib/tasks/job.rake 0 → 100644
    1 require "rubygems"
    2 require "open-uri"
    3 require "nokogiri"
    4 require "mechanize"
    5 require "pry"
    6
    7 namespace :Job do
    8 desc "Import data from crawler to database"
    9 task :import do
    10 puts "Starting import data to database"
    11 end
    • Son Do Hong @sondh commented Dec 05, 2019
      Master

      sau 1 block code thì em cách ra 1 dòng

      sau 1 block code thì em cách ra 1 dòng
    • Đường Sỹ Hoàng @hoangds

      changed this line in version 2 of the diff

      Dec 06, 2019

      changed this line in version 2 of the diff

      changed this line in [version 2 of the diff](https://gitlab.zigexn.vn/hoangds/VenJob/merge_requests/2/diffs?diff_id=4428&start_sha=f24e4f4bf352d2aae1c9a146774ce036efaf41f5#f7088fd37ab62eb9ccec3914041a48694bdadd6b_11_7)
      Toggle commit list
    Please register or sign in to reply
  • Đường Sỹ Hoàng @hoangds

    added 1 commit

    • 6d1348ad - second commit

    Compare with previous version

    Dec 06, 2019

    added 1 commit

    • 6d1348ad - second commit

    Compare with previous version

    added 1 commit * 6d1348ad - second commit [Compare with previous version](https://gitlab.zigexn.vn/hoangds/VenJob/merge_requests/2/diffs?diff_id=4428&start_sha=f24e4f4bf352d2aae1c9a146774ce036efaf41f5)
    Toggle commit list
  • Đường Sỹ Hoàng @hoangds

    changed title from First commit to Jobs Crawler

    Dec 06, 2019

    changed title from First commit to Jobs Crawler

    changed title from **{-First commit-}** to **{+Jobs Crawler+}**
    Toggle commit list
  • Van Hau Le
    @haulv started a discussion on the diff Dec 06, 2019
    Resolved by Van Hau Le Dec 06, 2019
    Gemfile
    45 49 end
    46 50
    47 51 group :test do
    48 52 # Adds support for Capybara system testing and selenium driver
    49 gem 'capybara', '>= 2.15'
    50 gem 'selenium-webdriver'
    53 gem "capybara", ">= 2.15"
    54 gem "selenium-webdriver"
    51 55 # Easy installation and use of web drivers to run system tests with browsers
    52 gem 'webdrivers'
    56 gem "webdrivers"
    53 57 end
    54 58
    55 59 # Windows does not include zoneinfo files, so bundle the tzinfo-data gem
    56 gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]
    60 gem "tzinfo-data", platforms: [:mingw, :mswin, :x64_mingw, :jruby]
    • Van Hau Le @haulv commented Dec 06, 2019
      Master

      @hoangds sao tốn time đi update toàn bộ cái string quote trong file gem này vậy?

      @hoangds sao tốn time đi update toàn bộ cái string quote trong file gem này vậy?
    • Đường Sỹ Hoàng @hoangds commented Dec 06, 2019
      Master

      em chỉnh cho đồng bộ với dấu " thôi anh.

      em chỉnh cho đồng bộ với dấu `"` thôi anh.
    Please register or sign in to reply
  • Đường Sỹ Hoàng @hoangds

    added 1 commit

    • b9c20920 - Deleted wheneverize

    Compare with previous version

    Dec 06, 2019

    added 1 commit

    • b9c20920 - Deleted wheneverize

    Compare with previous version

    added 1 commit * b9c20920 - Deleted wheneverize [Compare with previous version](https://gitlab.zigexn.vn/hoangds/VenJob/merge_requests/2/diffs?diff_id=4429&start_sha=6d1348ad9106c867e100c2f4cdbc1b66bdd408b8)
    Toggle commit list
  • Van Hau Le
    @haulv started a discussion on an old version of the diff Dec 06, 2019
    Resolved by Van Hau Le Dec 06, 2019
    lib/tasks/job.rake 0 → 100644
    27 description = job_page.css("div.content_fck.content_job_info").css("div.decs")[0].text
    28 requirement = job_page.css("div.content_fck.content_job_info").css("div.decs")[1].text
    29 elsif job_page.css("div.content_fck.job_requirement").present?
    30 description = job_page.css("div.content_fck.job_requirement").css("div")[0].text
    31 requirement = job_page.css("div.content_fck.job_requirement").css("div")[2].text
    32 elsif job_page.css("div.content_fck").present?
    33 description = job_page.css("div.content_fck")[0].css("p").text
    34 short_description = "N/A"
    35 requirement = job_page.css("div.content_fck")[1].css("p").text
    36 end
    37
    38 salary = job_page.css("div#showScroll.box2Detail").css("ul.DetailJobNew").css("li.bgLine2").css("p.fl_right").text
    39 position = job_page.css("div#showScroll.box2Detail").css("ul.DetailJobNew").css("li.bgLine1").css("p.fl_left").text
    40
    41 rescue
    42 puts "Skip #{link}"
    • Van Hau Le @haulv commented Dec 06, 2019
      Master

      @hoangds sử dụng rails logger nhé em! Không xài puts! Puts chỉ xài trong development env để debug thôi nhé!

      Edited Dec 06, 2019 by Van Hau Le
      @hoangds sử dụng rails logger nhé em! Không xài puts! Puts chỉ xài trong development env để debug thôi nhé!
    • Đường Sỹ Hoàng @hoangds

      changed this line in version 4 of the diff

      Dec 06, 2019

      changed this line in version 4 of the diff

      changed this line in [version 4 of the diff](https://gitlab.zigexn.vn/hoangds/VenJob/merge_requests/2/diffs?diff_id=4431&start_sha=b9c209206b0928cbd66603ce19e9fd33089fb346#f7088fd37ab62eb9ccec3914041a48694bdadd6b_41_41)
      Toggle commit list
    Please register or sign in to reply
  • Đường Sỹ Hoàng @hoangds

    added 1 commit

    • 9711ba07 - Rails logger

    Compare with previous version

    Dec 06, 2019

    added 1 commit

    • 9711ba07 - Rails logger

    Compare with previous version

    added 1 commit * 9711ba07 - Rails logger [Compare with previous version](https://gitlab.zigexn.vn/hoangds/VenJob/merge_requests/2/diffs?diff_id=4431&start_sha=b9c209206b0928cbd66603ce19e9fd33089fb346)
    Toggle commit list
  • Đường Sỹ Hoàng @hoangds

    added 1 commit

    • db5fb9fa - Fixed rail logger

    Compare with previous version

    Dec 06, 2019

    added 1 commit

    • db5fb9fa - Fixed rail logger

    Compare with previous version

    added 1 commit * db5fb9fa - Fixed rail logger [Compare with previous version](https://gitlab.zigexn.vn/hoangds/VenJob/merge_requests/2/diffs?diff_id=4433&start_sha=9711ba07dc2b6f34e1411f92b7ebd0c27a48fc2d)
    Toggle commit list
  • Son Do Hong @sondh

    mentioned in commit f5b0ef08

    Dec 06, 2019

    mentioned in commit f5b0ef08

    mentioned in commit f5b0ef082aeef0cc3c6e2fd110d215954fe58e34
    Toggle commit list
  • Son Do Hong @sondh

    merged

    Dec 06, 2019

    merged

    merged
    Toggle commit list
  • Write
  • Preview
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment
Son Do Hong
Assignee
Son Do Hong @sondh
Assign to
None
Milestone
None
Assign milestone
Time tracking
3
3 participants
Reference: hoangds/VenJob!2
×

Revert this merge request

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.
×

Cherry-pick this merge request

Switch branch
Cancel
A new branch will be created in your fork and a new merge request will be started.