Skip to content

[pull] master from jeremycw:master#5

Open
pull[bot] wants to merge 20 commits into
FreezeSoul:masterfrom
jeremycw:master
Open

[pull] master from jeremycw:master#5
pull[bot] wants to merge 20 commits into
FreezeSoul:masterfrom
jeremycw:master

Conversation

@pull

@pull pull Bot commented Feb 1, 2023

Copy link
Copy Markdown

See Commits and Changes for more details.


Created by pull[bot]

Can you help keep this open source service alive? 💖 Please sponsor : )

* Start implementation of ragel parser

* Add character limits for parsing

* Add generic error handling

* clean up unneccessary limit increments

* Clean up Makefile and test scripts

* Get unit test running

* Remove callback interface for parser

* Test chunked requests

* start integration with http_parser.c

* Fixed issues with chunked mode

* Major refactor

* Get major refactor compiling

* Use LLVM clang-format style

* Update docs

* Add Doxyfile

* Fix ragel file after clang-format mangled it

* Get unit tests running again

* Fix formatting on connection.c

* Fix failure to init parser bug

* Fix formatting of test_parser.c

* Clean up makefile

* Add state machine diagram uml

* More tests and fixes

* Add write_socket unit test

* Get integration tests running again

* Fix style errors

* Update httpserver.h with style fixes

* Start switch to cmake builds

* Update test includes

* Update cmake project name

* Clean up io_events.c

* Fix some issues with io_events cleanup

* Update procedure name and formatting

* Update read_socket.h formatting

* Use Makefile as command runner

* Update target names

* Get debug definition working

* Fix unit tests

* Get functional tests running

* Start using github actions

* Update httpserver.h

* Add cpp target

* Add auto generation notice to httpserver.h

* Dont use NOTE_SECONDS for portability

* Fix format error
@pull pull Bot added the ⤵️ pull label Feb 1, 2023
* Rename methods with better names

* Connection refactor

* Clean up write_socket procedure

* Fix common.h include issue
When _grwprintf's buffer needed to grow, the second vsnprintf call was
using the same consumed va_list args, causing undefined behavior and
corrupted response output.

Fix: properly va_end and va_start args around the reallocation.
…ovements

Fuzz testing:
- Add random_parser.c for random HTTP request fuzzing
- Add fuzz_parser.c for libfuzzer integration
- Add afl_parser.c for AFL-based fuzzing with seeds
- Add run_afl.sh script for AFL execution
- Add test/fuzz/CMakeLists.txt
- Add fuzz-random and fuzz-libfuzzer targets to Makefile
- Enable IWYU with corrected mapping file path

Functional test improvements:
- Add new test endpoints: /http10, /url-encoded, /notfound,
  /error, /custom-status, /connection-close, /connection-keep
- Update functional-test-runner with improved test cases
- Update expected results for new endpoints

Unit test improvements:
- Add HS_TEST_WRITE_CAPTURE mode for capturing response data
- Add hs_test_reset_capture() to reset capture buffer
- Expose hs_test_write_mode as non-static for cross-file access
- Add test_respond_large_body_capture test
- Add test_respond_grwprintf_truncation_bug test
- Add DEBUG compile definition for write() mocking
- Fix destroy_test_request to avoid double-free

Bug fix:
- Fix _grwprintf va_list consumption bug causing response corruption
  (va_end/va_start properly around buffer reallocation)

Cleanup:
- Remove compiled binary and test artifacts from repository
- Add patterns to .gitignore to prevent future commits
AFL requires special compiler instrumentation and has issues on macOS
(due to crash reporter and SIP). LibFuzzer is built into clang/LLVM
and works out of the box on macOS.

Changes:
- Remove afl_parser.c (AFL-specific harness)
- Remove run_afl.sh script
- Remove seeds/ directory
- Remove test/fuzz/CMakeLists.txt
- Simplify Makefile: single 'fuzz' target runs both libfuzzer
  and random fuzzer
- Add close(s) before exit(1) when bind() fails in _hs_bind_localhost
- Add missing #include <unistd.h> for close() declaration
- Fix Makefile fuzz target path issues
- Add standalone fuzz_harness.c for libfuzzer-style testing
Based on Ragel parser analysis, implements targeted fuzzing for:
- MUT_VALID: valid HTTP with bitflip mutations
- MUT_METHOD_BOUNDARY: long method names (limit is 32)
- MUT_INVALID_VERSION: HTTP/1.2+ versions
- MUT_CHUNKED_EDGE: chunked encoding edge cases
- MUT_CONTENT_LENGTH_EXTREME: large content lengths
- MUT_MALFORMED_HEADERS: malformed header parsing
- MUT_BUFFER_UNDERFLOW: incomplete requests
- MUT_EOF_MIDSTREAM: early EOF with Content-Length

Mutations: bitflip, delete, insert, truncate
Definitions of captured_write_buf, captured_write_size, and hs_test_write_mode
were in both the .c and .h file causing multiple definition errors on Linux/GCC.

Fix: Move all definitions to header, include header in .c file.
- Remove unnecessary #include <stdlib.h> from io_events.c (IWYU)
- Add missing #include <stddef.h> to test_write_socket.h (IWYU)
- Remove unnecessary #include <unistd.h> from test_respond.c (IWYU)
- Remove unnecessary #include <stdio.h> from test_request_util.c (IWYU)
- Fix multiple definition linker error: use extern for variable declarations in header, definitions only in test_write_socket.c
The test_respond_large_body_capture and test_respond_grwprintf_truncation_bug tests
were calling hs_request_begin_write which internally calls hs_request_terminate_connection
after write failure, freeing the request. The tests then called destroy_test_request
causing a double-free.

Fix: For capture tests, don't call hs_request_begin_write. The test_http_write
callback (which just counts writes) is used instead for the error_single_write
test only.

Also fix destroy_test_request ordering to set pointers to NULL after free.
The capture tests (large_body_capture, grwprintf_truncation_bug) were calling
hs_request_begin_write which would call hs_request_terminate_connection
on write error, freeing the request. But the tests then tried to use the
captured buffer which was now owned by the request.

Fix: Have test_http_write callback also capture bytes when in CAPTURE mode
by calling hs_test_write directly. This allows proper cleanup via
destroy_test_request without double-free.

Also adds hs_test_write declaration to test_write_socket.h for proper
extern access.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant